Asymptotic behaviour of $\sum_{p\leq x} \frac{1}{p^2}$

Using an estimate of the difference between the prime counting function, $\pi(n)$ and the logarithmic integral function, $\mathrm{li}(x)$, we can estimate the tail of the series.

Let $\Delta(n)=\pi(n)-\mathrm{li}(n)$. Without assuming the Riemann Hypothesis, we have that for any $m$ $$ \Delta(n)=O\left(\frac{\raise{2pt}n}{\log(n)^m}\right)\tag{1} $$

Summing by parts and using $(1)$ with $m=2$, we get $$ \begin{align} \sum_{k=n}^\infty\frac{1}{k^2}(\Delta(k)-\Delta(k-1)) &=-\frac{\Delta(n-1)}{n^2}+\sum_{k=n}^\infty\Delta(k)\left(\frac{1}{k^2}-\frac{1}{(k+1)^2}\right)\\ &=O\left(\frac{1}{n\log(n)^2}\right)\tag{2} \end{align} $$ Using $(2)$, we get $$ \begin{align} \sum_{p\ge n}\frac{1}{p^2} &=\sum_{k=n}^\infty\frac{1}{k^2}(\pi(k)-\pi(k-1))\\ &=\sum_{k=n}^\infty\frac{1}{k^2}(\mathrm{li}(k)-\mathrm{li}(k-1))+O\left(\frac{1}{n\log(n)^2}\right)\\ &=\int_n^\infty\frac{\mathrm{d}x}{x^2\log(x)}+O\left(\frac{1}{n\log(n)^2}\right)\\ &=\int_n^\infty\frac{\log(x)+1}{x^2\log(x)^2}\mathrm{d}x+O\left(\frac{1}{n\log(n)^2}\right)\\ &=\frac{1}{n\log(n)}+O\left(\frac{1}{n\log(n)^2}\right)\tag{3} \end{align} $$

The prime zeta function $P(s)$, for $\text{Real}(s) > 1$, is defined as $$P(s) = \sum_{\overset{p=1}{p \text{ is prime}}}^{\infty} \dfrac1{p^s}$$ The sum converges for $\text{Real}(s) > 1$, similar to the $\zeta$-function. Your sum is $P(2)$ and is approximately $0.4522474200410654985065\ldots$.

There are no "nice" values for $P(s)$ where $s \in \mathbb{Z}^+ \backslash \{1\}$. A very crude argument why there are no "nice" values for $P(s)$ is due to the fact that the function, $$g(n) = \dfrac{\mathbb{I}_{n \text{ is a prime}}}{n^s}$$ is not a "nice" arithmetic function in the usual sense i.e. it is not even multiplicative for instance.