$x(p − x)y(p − y)$ is square implies $x = y$

This is the problem from the second stage of last year's Polish Mathematical Olympiad, see here. Below I present the official problem solution, from here.

Suppose the expression is a square, say $k^2$. Applying AM-GM inequality we quickly find $k<p^2/4$. On the other hand, from $x,y<p/2$, we find $k>xy$. Now we rewrite $$0=k^2-xy(p-x)(p-y)=(k-xy)(k+xy)-pxy(p-x-y),$$ from where $p\mid k-xy$ or $p\mid k+xy$.

Suppose first $k-xy=lp>0$. Then $$lp(lp+2xy)=lp(k+xy)=pxy(p-x-y),\\ p(xy-l^2)=xy(2l+x+y).$$ Since $x,y<p$, $p\mid 2l+x+y$. From $lp=k-xy<k<p^2/4$ we get $l<p/4$, and then $2l+x+y<2p/4+p/2+p/2=2p$, hence $2l+x+y=p$. However, then $p(xy-l^2)=pxy$, impossible given $l>0$.

Hence $k+xy=lp$ for some $l$. Like above, we get $p(xy-l^2)=xy(x+y-2l)$. From $lp=k+xy<p^2/4+(p/2)^2=p^2/2$ we get $l<p/2$. Hence $$p>x+y>x+y-2l>-2\cdot p/2=-p,$$ thus $x+y-2l=0$ (since it's divisible by $p$). Thus $p(xy-l^2)=0$, i.e. $xy=l^2$. Hence finally $$(x-y)^2=(x+y)^2-4xy=(2l)^2-4l^2=0$$ giving $x=y$.

$gcd(x,y)=n_1,gcd(x,p-y)=n_2$, $y$ and $p-y$ coprime, so $gcd(x,y(p-y))=n_1n_2$

Let $x=n_1n_2x',y=n_1y',p-y=n_2y''$, we have $x'(p − x)y'y''$ is square, so $x'$ is square.

In similar fashion, we can get: $x=n_1n_2a^2,y=n_1n_3c^2,p-x=n_3n_4b^2,p-y=n_2n_4d^2$

Now we have:

$p = n_1n_2a^2+n_3n_4b^2=n_1n_3c^2+n_2n_4d^2\tag1$

This can be solved in similar fashion as this answer.

To be complete:

$p^2=(n_1n_2a^2+n_3n_4b^2)(n_1n_3c^2+n_2n_4d^2)$

so

$p^2=n_2n_3(n_1ac+n_4bd)^2+n_1n_4(n_2ad-n_3bc)^2\tag2$

$p^2=n_2n_3(n_1ac-n_4bd)^2+n_1n_4(n_2ad+n_3bc)^2\tag3$

By(1), we have

$(p - n_1n_2a^2)n_1n_2d^2=(p - n_1n_3c^2)n_1n_3b^2$

which implies

$p(n_3b^2-n_2d^2)=n_1(n_2ad-n_3bc)(n_2ad+n_3bc)\tag4$

From(4), $p$ divides $(n_2ad-n_3bc)$ or $(n_2ad+n_3bc)$. If $p$ divides $(n_2ad-n_3bc)$, then from (2) we get $(n_2ad-n_3bc)=0$, so from (4) $n_3b^2-n_2d^2=0$, so $p-x=p-y$. If $p$ divides $(n_2ad+n_3bc)$,then from (3) we get $(n_1ac-n_4bd)=0$, so $n_1ac=n_4bd$. $a$ and $b,d,n_4$ are coprime, so $a=1$.similarly $b=c=d=1$. Then $n_1=n_4=1$,$x=n_2,y=n_3, p=n_2+n3$, this contradicts $x,y\le\frac{p-1}{2}$.