Word which means that a number has magnitude greater than one?

Have you considered symbolic syntax? For coefficients - a math concept - I'd think something like |β| > 1 (absolute value greater than 1) would be appropriate.

If symbolic isn't an option, I'd suggest "there were a number of scenarios with a non-directional coefficient greater than one". It's not open to ambiguity and works as an adjective to coefficient instead of introducing a new concept like "coefficient with a ..." does.

It would be correct to say "...coefficients outside the closed interval [-1,1]." I'm pretty sure there's no specific adjective to modify "coefficients" that does what you want, but if I had to make one up it would be "superunital."