Rate of convergence of series of squared prime reciprocals

Lets rearrange the sum $C-\sum_{p\leq x}\frac{1}{p^{2}}=\sum_{p>x}\frac{1}{p^{2}}$. Using integration by parts this is $$\sum_{p>x}\frac{1}{p^{2}}=\int_{x}^{\infty}\frac{1}{t^{2}}d\left(\pi(t)\right)=\frac{\pi(t)}{t^{2}}\biggr|_x^\infty+2\int_{x}^{\infty}\frac{\pi(t)}{t^{3}}dt.$$ Using the prime number theorem, that is the asymptotic for $\pi(x),$ you can deduce that the quantity on the right hand side is $\sim\frac{1}{x\log x},$ which is your rate of convergence.

Just worth noting, this is exactly what we would expect. The tail of the sum over all integers has size $\frac{1}{x}$, that is $\sum_{n>x} \frac{1}{n^2}\sim \frac{1}{x}$ and the primes occur with density $\frac{1}{\log n}$ around $n$, so we would expect the tail of the sum to be of size $\frac{1}{x\log x}$. Partial summation/integration allows to prove this.

Edit: Replaced $\asymp$ with $\sim$, since as pointed out by Greg Martin in the comments, the Prime Number Theorem is strong enough to yield this/