A *finite* first order theory whose finite models are exactly the $\Bbb F_p$?
Solution 1:
EDIT: This answer is only a proof of a special case of the given question (the case when the infinite models of the given theory are exactly the infinite fields). It does not solve the general question.
Theorem. Let $\mathcal{M}$ be the class of all fields $F$ such that either $F$ is infinite or $F$ is isomorphic to $\Bbb{F}_p$ for some prime $p$. Then $\mathcal{M}$ has no finite FOL-axiomatization.
Proof. For any prime $p$ and any integer $n \geq p$, let $\sigma_{p,n}$ be the translation of the following statement of ring theory into a FOL-sentence:
If $R$ has cardinality $n$ and $p \cdot 1_R = 0_R$, then for every ring element x, x is either $0_R$, $1_R$, $2 \cdot 1_R$, $\ldots$, or $(p-1) \cdot 1_R$.
Let $\Sigma$ be the union of the first-order field axioms and the set of all FOL-sentences $\sigma_{p,n}$ for some prime number $p$ and for some integer $n \geq p$. Then the class of all models of $\Sigma$ is clearly $\mathcal{M}$.
Suppose now that the class $\mathcal{M}$ has a finite FOL-axiomatization $\{ \rho_1, \ldots, \rho_m \}$. Let $\rho$ be the conjunction of the sentences $\rho_i$ for $i = 1, \ldots, m$. Then since $\Sigma$ is also an axiomatization of $\mathcal{M}$, $\rho$ follows logically from $\Sigma$. Thus, $\rho$ follows logically from a finite subset $\Sigma_0$ of $\Sigma$ by the compactness theorem of first-order logic. Then $\Sigma_0$ is a finite axiomatization of the class $\mathcal{M}$.
Since $\Sigma_0$ is finite and there are infinitely many primes, we can choose a prime $\hat{p}$ and an integer $\hat{n} \geq \hat{p}$ such that for every prime $p \geq \hat{p}$ and for every $n \geq \hat{n}$, $\sigma_{p,n}$ is not an element of $\Sigma_0$. Then, clearly, every field $\Bbb{F}_{p^n}$ for $p \geq \hat{p}$ and $n \geq \hat{n}$ is a model of $\Sigma_0$ and must therefore be an element of $\mathcal{M}$. But this contradicts our initial assumption for $\mathcal{M}$. Therefore $\mathcal{M}$ has no finite FOL-axiomatization.
Q.E.D.