Can an algebraic structure have indistinguishable elements?

Solution 1:

Consider the field $\Bbb Q(t,s)$ where $t,s$ are two algebraically independent transcendental numbers.

Then these two numbers are completely inseparable by a first-order formula in the language of fields.

Generally speaking, if $\cal L$ is some first-order language of some structure, then there are at most $\aleph_0\cdot|\cal L|$ definable elements in any given structure. If by "indistinguishable" we mean "inseparable by a first-order formula with limited parameters$^1$", then any sufficiently large structure will invariably contain a lot of indistinguishable elements.

One good place to learn about these things is model theory, and in particular the concept of "type".

Edit: To your last edit, about $(\Bbb N,f)$ note that $0$ is a definable element of the structure with the formula $x=f(x)$. And since we don't have any other symbols in the language it's really impossible to express anything else. Therefore it's very easy to see that over the empty set, every two non-zero elements satisfy the same formulas with one free variable.

(To see that we can't express anything else, at least without parameters, note that if $m,n$ are non-zero then there is an automorphism which exchanges between the two. Therefore every two non-zero elements must satisfy the same formulas [in one free variable].)

Footnotes:

1. Of course if we allow any parameter then $\varphi(x,y)$ defined as $\lnot(x=y)$ is sufficient to distinguish between any two members. But if, like in the first example, we allow no parameters - or parameters from a small substructure - then if the universe of the structure is large enough, there will be many indistinguishable elements.

Solution 2:

A category usually has distinct but isomorphic objects. This generalizes both of your bulleted examples: a preorder is a category in which there is at most one morphism between any two objects, and a pseudometric space is an enriched category (see Lawvere metric space). Getting rid of this extra ambiguity amounts to taking a skeleton.

Since categories are ubiquitous, this gives a wealth of examples. Here are some which are more algebraic in flavor:

• A set $X$ together with an action of a group $G$ may be regarded as a category (in fact a groupoid) with a morphism $x \to y$ for every $g \in G$ such that $gx = y$. Two objects are isomorphic iff they are in the same orbit with respect to the group action.
• Given a field $k$ we can consider the category of algebraic extensions of $k$. This category contains various algebraic closures of $k$, all of which are isomorphic.