What is the center of a semidirect product?

I'm afraid the answer by Alexander is wrong. The given subgroup is indeed contained in the center of the semidirect product, but the reverse inclusion can fail.

Simple example: let $G_1$ be a center-free group, with an element $x$ of prime order $p$ (for instance, pick $G_1$ non-abelian of order 6 and $p=2$ or 3). Let the cyclic group $G_2$ of order $p$ act on $G_1$ so that the generator $c$ of order $p$ acts by conjugation by $x$: $\varphi(c^n)(g)=x^ngx^{-n}$. Then $\varphi:G_2\to\mathrm{Aut}(G_1)$ is injective, and hence the subgroup $(Z(G_1)\cap \mathrm{Fix}(\varphi))\times(Z(G_2)\cap\mathrm{Ker}(\varphi))$ is trivial. But the center of the semidirect product $G=G_2\rtimes G_1$ is not trivial: if the law of the latter is given by $(g_2,g_1)(h_2,h_1)=(g_2\varphi(g_1)(h_2),g_1h_1)$, then the element $(x,c^{-1})$ is central (thus the center $Z$ is cyclic of order $p$ and $G$ is direct product of $G_1$ and $Z$).

A correct description of the center of an arbitrary semidirect product should involve the kernel of the map $\varphi':G_2\to\mathrm{Out}(G_1)$, but it looks a bit complicated.

Edit: the center can be described as follows: let $f$ be the canonical map $G_2\to \mathrm{Inn}(G_2)=G_2/Z(G_2)$ and $s$ the canonical map $\mathrm{Ker}(\varphi')\to\mathrm{Inn}(G_2)$. Then the center of $G$ is the set of pairs $(g_2,g_1)\in G_2\rtimes G_1$ such that $g_1\in s^{-1}(f(\mathrm{Fix}(\varphi)))\cap Z(G_1)$, and $f(g_2)=s(g_1)^{-1}$.

Denote by $\varphi_g$ the automorphism of $G_1$ associated with a $g\in G_2$. Take $z\in \operatorname{Z}(G_1\rtimes G_2)$ and write $z=xy$ for $x\in G_1$ and $y\in G_2$. Then, for every $g\in G_2$, we have $$gxy=\varphi_g(x)gy=xyg.$$ By equating the $G_1$ and $G_2$ parts in $gxy=\varphi_g(x)gy$, we see that $x=\varphi_g(x)$. Similarly, we equate the $G_1$ and $G_2$ parts in $gxy=xyg$ to obtain $gy=yg$. Since this is true for all $g\in G_2$, we may conclude that $x\in \operatorname{Fix}(\varphi)$ and $y\in \operatorname{Z}(G_2)$. With this in mind, we revisit our earlier computation: $$gxy=yxg=\varphi_y(x)yg=xyg$$ Thus we see that $x=\varphi_y(x)$, so $\varphi_y$ is the identity automorphism, and we have that $y\in \operatorname{Ker}(\varphi)$. Finally, taking $g\in G_1$, we have that $gxy=xyg$, from which we see that $x\in Z(G_1)$.

Putting it all together, we have $\operatorname{Z}(G_1\rtimes G_2) \subseteq \left(\operatorname{Z}(G_1)\cap\operatorname{Fix}(\varphi)\right)\rtimes \left(\operatorname{Ker}(\varphi)\cap \operatorname{Z}(G_2)\right),$ which is simply $\left(\operatorname{Z}(G_1)\cap\operatorname{Fix}(\varphi)\right)\times \left(\operatorname{Ker}(\varphi)\cap \operatorname{Z}(G_2)\right)$. The reverse inclusion is not true in general, as can be seen in YCor's answer below. So, all we can really say at this point is that $$\left(\operatorname{Z}(G_1)\cap\operatorname{Fix}(\varphi)\right)\times \left(\operatorname{Ker}(\varphi)\cap \operatorname{Z}(G_2)\right)\subseteq \operatorname{Z}(G_1\rtimes G_2)$$ I would be curious to see what additional description could bring this to an equality.