Leaving out one of the Peano Axioms
What happens if you leave N4 (from Ross' book) out of the Peano axioms which states that if $n$ and $m$ in $\mathbb{N}$ have the same successor, then $n = m$?
This is the axiom that forces $\mathbb{N}$ to be infinite; if you drop it then you lose the requirement that your 'set of natural numbers' be infinite.
$\mathbb{N}$ would satisfy the remaining axioms, but so would the set $\{ 0, 1 \}$, where we define $\mathtt{succ}(0)=1$ and $\mathtt{succ}(1)=1$. In fact any set of the form $\{ 0, 1, \cdots, n \}$ (for $n \ge 1$) would satisfy the remaining Peano axioms if we define $\mathtt{succ}(k)=k+1$ for $k < n$ and $\mathtt{succ}(n)=n$.
If you omit the axiom $\, Sm = Sn\,\Rightarrow\, m=n\,$ then $S$ need no longer be $1$-$1,\,$ so the axioms now admit finite "dipper" semigroup models, where the orbit of $S$ starting from $0$ need not be an infinite half-line, but may be shaped like the big dipper (i.e. shaped like the letter $\rho),\,$ with an initial preperiod followed by a periodic part. If $\, S^{j+k}0 = S^j0\,$ with $\,j,k\,$ minimal then $\,0,S0,\ldots,S^{j-1}0\, $ is the preperiod, and a periodic cycle of length $\,k\,$ starts at $\,S^j0 = S^{j+k}0 = S^{j+2k}0 =\, \ldots \,$