Geometric interpretation of the fundamental theorem for coalgebras?

Very interesting. Unfortunately the dual statement you suggest is false. Since $k$ is a field, $k [x]$ is a principal ideal domain. In particular, any non-trivial quotient of $k [x]$ is automatically finite-dimensional. Now, any linear map $f : k [x] \to k$ is freely and uniquely determined by the sequence $f(1), f(x), f(x^2), \ldots$, and $f$ factors through the quotient $k [x] / (p(x))$ only if $$a_0 f(x^n) + a_1 f(x^{n+1}) + \cdots + a_d f(x^{n+d}) = 0$$ for all $n$, where $$p(x) = a_0 + a_1 x + \cdots + a_d x^d$$ so to find an $f$ that does not factor through any non-trivial quotient of $k [x]$, it is enough to find a sequence that does not satisfy any linear recurrence equation (with constant coefficients). But a sequence that satisfies a linear recurrence equation grows at worst exponentially, so we can just take a super-exponential sequence, say $n!$.

Perhaps the following observation is of use. The fundamental theorem of coalgebras can be stated as establishing an equivalence of categories between $Coalg$ and the category $Ind-coalg$. (I will use lower case $c$ in the notation to indicate finite dimensional ones. Here $Ind-coalg$ is the category of 'ind-objects' in $coalg$.) The duality you mention is between $coalg$ and $alg$, so we expect (and obtain) a dualtiy between $Coalg$ and $Pro-alg$, the category of `pro-objects' in $alg$. Under reasonable assumptions on the ground ring, these correspond to pseudo-compact algebras. This leads to two further points.

(i) When dealing with duals in this context, you need to have some continuity assumptions around to get the dual conditions for the coalgebra case.

(ii) For the geometric interpretation note that Grothendieck developed the theory of formal groups using pseudocompact rings and also that for any profinite group, (e.g. Galois groups and algebraic analogues of the fundamental group of a space), the natural analogue of the group algebra is a pseudocompact ring.

EDIT: In this answer all coalgebras are cocommutative and all algebras commutative

As was already mentioned the statement you mention is equivalent to the statement that there's an equivalence $CoAlg \cong Ind(CoAlg_{fin})$ (everything is over a fixed field $k$).

We can make this statement more geometric by noticing that

$$Ind(Coalg_{fin}(k)) \cong Flat(CoAlg_{fin}^{op},Set) \cong Flat(Alg_{fin},Set) \cong Flat(Aff_{fin}^{op},Set)$$

Where "$Flat(-,-)$" means flat functors i.e. functors preserving finite limits. This last category is equivalent to the category of pro-representable functors on artin rings or in more geometric terms formal schemes (which are set-theoretically supported on a discrete set of points).

In summary the category of Coalgebras over $k$ is equivalent to the category of Formal Schemes (ind-finite over $k$).