The set of all infinite binary sequences

You were supposed to be assuming, for the sake of contradiction, that $S$ was countably infinite; Cantor's diagonal argument tells you how to construct, for any countably infinite collection of binary sequences, a binary sequence not in that collection. Thus, the set $S$ of all binary sequences (which is a perfectly well-defined object) is uncountable.

To create a new set not in the collection, Cantor's argument starts as "Choose first element of new set such that it differs at first position from first set in collection, then choose second element for new set such that it differs at second position from second set in collection and so on...".

The fact that it uses is that you have a map from $\Bbb N\to A$ (the collection of all sequences(sets)), so it assumes that collection to be countable. But, since this argument contradicts the completeness of the collection , therefore, it implies that you can't have any map from $\Bbb N\to A$ and thus, $A$ is uncountable.

You can define $A$ or $S$ as $$\{(s_1,s_2,s_3,\cdots): s_i\in\{0,1\} \forall i\in \Bbb N\}$$

The only thing that cantor's diagonal argument prove is that this set of sequence is uncountable .