How to prove that a function on a quotient set $\mathbb{Z}/n\mathbb{Z}$ is well defined?
Hint:
If you have $[a]=[b]$ then you need to prove that $[a^2]=[b^2]$ too.
Remember that $[a]=[b]$ implies that $a-b$ is a multiple of the modulus $n$, that is, $a-b=kn$ for some integer $k$.