Prove that $\lfloor \sqrt{p} \rfloor + \lfloor \sqrt{2p} \rfloor +...+ \lfloor \sqrt{\frac{p-1}{4}p} \rfloor = \frac{p^2 - 1}{12}$

Problem
Prove that $\lfloor \sqrt{p} \rfloor + \lfloor \sqrt{2p} \rfloor +...+ \lfloor \sqrt{\frac{p-1}{4}p} \rfloor = \dfrac{p^2 - 1}{12}$ where $p$ prime such that $p \equiv 1 \pmod{4}$.

I really have no idea how to start :(! The square root part really messed me up. Can anyone give me a hint?

Thank you

The sum $S(p)$ counts the lattice points with positive coordinates under $y=\sqrt{px}$ from $x=1$ to $x=\frac{p-1}{4}$. Instead of counting the points below the parabola, we can count the lattice points on the parabola and above the parabola, and subtract these from the total number of lattice points in a box. Stop here if you only want a hint.

Since $p$ is prime, there are no lattice points on that parabola (with that range of $x$ values).

The total number of lattice points in the box $1 \le x \le \frac{p-1}4, 1\le y \le \frac {p-1}2$ is $\frac{(p-1)^2}8$.

The lattice points above the parabola are to the left of the parabola. These are counted by $T(p)= \lfloor 1^2/p \rfloor + \lfloor 2^2/p \rfloor + ... + \lfloor (\frac{p-1}2)^2/p \rfloor$.

$T(p)+S(p) = \frac{(p-1)^2}8$, so $S = \frac {p^2-1}{12}$ is equivalent to $T(p) = \frac{(p-1)(p-5)}{24}$.

Consider $T(p)$ without the floor function. This sum is elementary:

$$\sum_{i=1}^{(p-1)/2} \frac{i^2}p = \frac 1p \sum_{i=1}^{(p-1)/2} i^2 = \frac 1p \frac 16 (\frac{p-1}2)(\frac {p-1}2 + 1)(2\frac{p-1}2 +1) = (p^2-1)/24.$$

What is the difference between these? Abusing the mod notation, $\frac{i^2}p - \lfloor \frac{i^2}p \rfloor = 1/p \times (i^2 \mod p)$. So,

$$(p^2-1)/24 - T(p) = \sum_{i=1}^{(p-1)/2} \frac{i^2}p - \lfloor \frac{i^2}p \rfloor = \sum_{i=1}^{(p-1)/2} \frac 1p \times (i^2 \mod p) = \frac 1p \sum_{i=1}^{(p-1)/2} (i^2 \mod p).$$

Since $i^2 = (-i)^2$, this last sum is over the nonzero quadratic residues. Since $p$ is $1 \mod 4$, $-1$ is a quadratic residue, so if $a$ is a nonzero quadratic residue, then so is $p-a$. Thus, the nonzero quadratic residues have average value $p/2$ and the sum is $\frac{(p-1)}2 \frac p2$.

$$(p^2-1)/24 - T(p) = \frac 1p \frac{(p-1)}2 \frac p2 = \frac{p-1}4$$ $$T(p) = \frac{(p-1)(p-5)}{24}.$$

That was what we needed to show.