Example of a model in set theory where the axiom of extensionality does not hold?

The axiom of extensionality says:

$$\forall x\forall y\left(x=y\leftrightarrow \forall z\left(z\in x\leftrightarrow z\in y\right)\right)$$

Obviously, if two sets are the same then they have the same elements. So in order to violate this axiom we need to have different sets which the model would think have the same elements.

If you just want a model of sets in which the axiom of extensionality does not hold, consider for $a\neq b$ the following: $$\left(U=\Big\{\{a,b\},\{a\},a\Big\}, \in\right)$$

We have that $a\in\{a\}$, and $a\in\{a,b\}$. Since $a\neq b$ we have that $\{a\}\neq\{a,b\}$, however for all $x\in U$ we have $x\in\{a\}\leftrightarrow x\in\{a,b\}$.

This is because $U$ does not know about $b$. It just knows that $\{a,b\}$ and $\{a\}$ are two distinct beings. It is unable to say why, in terms of $\in$ relation.

The problems begins when you want more axioms. The more axioms you would want to have, the more complicated your universe will have to get.

(This answer does not construct a model of ZFC - Extensionality. Instead, it discusses the purpose of the axiom of extensionality and its relationship to first order logic with equality.)

The point of the axiom of extensionality is to avoid the situation where the way in which the set is defined, not just its members, affects which sets contain the set. So to make extensionality fail you will just need a model in which there are two sets $A,B$ which have the same elements but are not equal.

The easiest way to do this is to take any set $A$, make a copy of it, color one copy red, and color the other copy blue. Declare that any set that was a member of $A$ is a member of both red-$A$ and blue-$A$, and that any set which contained $A$ contains both red-$A$ and blue-$A$. Then the resulting thing will still be a model of ZFC in the language with just $\in$ (without $=$ yet).

The point of the axiom of extensionality is then to say that this situation does not occur: there are not two sets which are somehow different despite having all the same elements.

In fact, a modification of the axiom of extensionality lets us define $=$ in terms of $\in$, by declaring that two sets will be regarded as equal if they have all the same members. To make this work, we rewrite the axiom of extensionality without $=$ as:

$$ (\forall x)(x \in A \Leftrightarrow x \in B) \Rightarrow (\forall y)(A \in y \Leftrightarrow B \in y) $$ Then we can define $$ A = B \Leftrightarrow (\forall x)(x \in A \Leftrightarrow x \in B) $$ and we can prove the substitution axioms for equality in terms of the equality-free axiom of extensionality and the definition of $=$.

This can be done in any model of ZFC in the language with just $\in$ (satisfying the modified version of extensionality) to obtain a model of ZFC in the language ($\in$, $=$), including the usual axiom of extensionality. But here $=$ may not be interpreted as the true equality relation, for example if the original model had red sets and blue sets. If we want the $=$ symbol to be interpreted as actual equality, we have to mod out by the equivalence relation induced by the interpretation of $=$ in our model. In the setting where $=$ is treated as a logical symbol which must be interpreted as true equality, the point of the axiom of extensionality is to ensure that this modding-out has already been performed: if we would define to sets to be "equal" in this indirect way, then they are already equal.

I haven't checked all the details, but I expect it would work simply to take a model of ZFC and then remove the empty set from the universe. All other sets and their membership relations stay unchanged. The new model still has an empty set (namely the one that was formerly $\{ \emptyset\}$), and I think all of the other axioms except extensionality would still hold, though verifying this could be tedious.

However, the two individuals that used to be $\{\{\emptyset\}\}$ and $\{\emptyset,\{\emptyset\}\}$ now have the same elements -- but the set that used to be $\{\{\{\emptyset\}\}\}$ contains one but not the other, violating extensionality.