Transitive sets: problem in proof of Lemma I.8.6 of Kunen's 'Foundations of Mathematics'

The assumption is that $\alpha$ is not any transitive set, but an ordinal. Which is a transitive set that is well-ordered by $\in$. In particular $\in$ is a transitive relation on $\alpha$ in the usual sense.

You are correct that this reasoning need not apply to arbitrary transitive sets, though.

I agree with you that the end of the proof of Lemma I.8.6 in Kunen is misleading and incomplete. It can be fixed as below.

Let $\alpha$ be an ordinal, that is, $\alpha$ is a transitive set (every element of it is a subset of it) such that the relation $\in$ on $\alpha$ is a well-order (i.e., a (strict) total order for which every nonempty subset has a minimal element).

At the end of the proof of I.8.6 we have $x\in y\in z\in\alpha$ with $x,y,z\in\alpha$. We have to show that $x\in z$. Because $\in$ is a total order, we either have (1) $z=x$, or (2) $z\in x$, or (3) $x\in z$. Case (1) is impossible because that would give a cycle $x\in y\in x$ with no minimal element for the set $\{x,y\}$. Case (2) is impossible because that would give a cycle $x\in y\in z\in x$ with no minimal element for the set $\{x,y,z\}$. So case (3) $x\in z$ must hold.

Your example $A=\{\emptyset, \{\emptyset\},\{\{\emptyset\}\}\}$ is a transitive set, but the $\in$ relation on $A$ is not a well-order, as it is not even a total order: the elements $\emptyset$ and $\{\{\emptyset\}\}$ are not comparable, as neither is a member of the other. As you correctly observed, $A$ being a transitive set is not enough to conclude that the relation $\in$ on $A$ is a transitive relation.