Solution 1:

Here's an explanation. It is easier to follow, if you draw a picture. Let $\vec{u}=(dx,dy)$ be the vector from the point $P_0=(x_0,y_0)$ to the point $P_1=(x_1,y_1)$, i.e. a vector pointing in the direction of the mirror line. Then $\vec{n}=(-dy,dx)$ is perpendicular to it. Let's name the vector from $P_0$ to $P$ $\vec{v}=(p.x-x_0,p.y-y_0)$. The projection of the vector $\vec{v}$ along the normal $\vec{n}$ is then $$\vec{p}=\frac{\vec{n}\cdot\vec{v}}{\vec{n}\cdot\vec{n}}\vec{n}=\frac{-(p.x-x_0)dy+(p.y-y_0)dx}{dx^2+dy^2}\vec{n}.$$

We compute the mirror image point $P'=(x_2,y_2)$ by comparing the representations of the vectors $\vec{P_0P}=\vec{v}$ and $\vec{P_0P'}=\vec{v'}$ in the (orthogonal) basis $\{\vec{u},\vec{n}\}$. The reflection maps $\vec{u}$ to itself, but the vector perpendicular to mirror is mapped to its negative: $\vec{n}\mapsto -\vec{n}$. Therefore the reflection maps $$ \vec{v}\mapsto\vec{v}'=\vec{v}-2\vec{p}. $$ The rest amounts to just plugging in the numbers. Write $N=dx^2+dy^2$ for short. We get $$ \begin{align} \vec{v}'&=\frac1N\left(N(p.x-x_0)-2dy^2(p.x-x_0)+2dy\,dx(p.y-y_0)\right)\vec{i}\\ &+\frac1N\left(N(p.y-y_0)+2dy\,dx(p.x-x_0)-2dx^2(p.y-y_0)\right)\vec{j}\\ &=\left(a(p.x-x_0)+b(p.y-y_0),b(p.x-x_0)-a(p.y-y_0)\right). \end{align} $$ Your formula comes from the fact that the above vector $\vec{v'}$ is the separation vector from $P_0$ to $P'$. Therefore we need to add the coordinates of $P_0$ to the result.

Edit: A crude image here. The points $P,P_0,P_1,P'$ are marked with the dots, and the vectors are $u,n,v,p,v',-2p$. Sorry about the missing arrows on top of those letters - can't do any better :-(

enter image description here