How to prove Euler's pentagonal theorem? Some hints will help

While there is a lot of value to the different bijective proofs known for Euler's pentagonal theorem, perhaps the proof that's easiest to see without having to draw pictures is Euler's original idea.

Define $A_N=\sum_{n=1}^{\infty}q^{N(n-1)}(1-q^N)\cdots (1-q^{N+n-1})$. Start with the identity $$\prod_{n=1}^{\infty}(1-a_n)=1-a_1-\sum_{n=2}^{\infty}a_n(1-a_1)\cdots (1-a_{n-1})$$ and plug in $a_n=q^n$. This gives you $\prod_{n=1}^{\infty}(1-q^n)=1-q-q^2A_1$. Now to finish the proof of Euler's theorem you need to prove the following recursion $$A_N=1-q^{2N+1}-q^{3N+2}A_{N+1}.$$ This identity is equivalent to proving $$q^{2N+1}-1+\sum_{n=1}^{\infty}q^{N(n-1)}(1-q^{N+1})\cdots(1-q^{N+n-1})(1-q^N+q^{3N+n+1}-q^{4N+2n+1})=0$$ and it starts $$(q^{2N+1}-1)+(1-q^N+q^{3N+2}-q^{4N+3})+q^N(1-q^{N+1})(1-q^N+q^{3N+3}-q^{4N+5})+\cdots$$ $$=q^N(1-q^{N+1})(q^{2N+2}-1)+q^N(1-q^{N+1})(1-q^N+q^{3N+3}-q^{4N+5})+\cdots$$ $$=q^{2N}(1-q^{N+1})(q^{2N+3}-1)+\cdots$$ and if you've noticed the pattern, filling out the details shouldn't be very hard.

Here is a somewhat combinatoric proof. Note that for $k>0$, the coefficient of $q^k$ in $$ \prod_{n=1}^\infty(1-q^n) $$ is the number of even, increasing partitions of $k$ minus the number of odd, increasing partitions of $k$. Take $k=7$, for example. There are $3$ even, increasing partitions of $7$: $1+6$, $2+5$, and $3+4$; there are $2$ odd, increasing partitions of $7$: $7$ and $1+2+4$. Thus, the coefficient of $q^7$ is $3-2=1$.

In what follows, we will only be concerned with increasing partitions. There is a nice way to cancel the even partitions with the odd partitions, but it fails in a few cases. These failing cases are what give rise to the non-zero terms in the summation.

Here is the way to pair even and odd partitions. Suppose the smallest summand in a partition is $m$.

1. if the $m$ largest summands are consecutive, then remove $m$ and add $1$ to the $m$ largest summands.
2. if only the $j$ largest summands are consecutive $(j<m)$, then subtract $1$ from the $j$ largest summands and prepend $j$.

Note that (1) is the inverse of (2) and vice-versa. Take $k=6$, for example. There are $4$ partitions, and they are paired as follows: $6\leftrightarrow 1+5$ and $1+2+3\leftrightarrow 2+4$.

Each of the two pairings above fail in exactly one way for each $m>0$,

1. for the partition $m+(m+1)+...+(2m-1)$ [$m$ terms totaling $(3m-1)m/2$]
   (this fails because $m$ is to be removed, yet it is also to be incremented)
2. for the partition $(m+1)+(m+2)+...+2m$ [$m$ terms totaling $(3m+1)m/2$]
   (this fails because $m$ is to be prepended, yet $m+1$ is also to be decremented)

Thus, after cancellation, we have leftover partitions contributing $(-1)^m q^{(3m-1)m/2}$ and $(-1)^m q^{(3m+1)m/2}$ for $m>0$. Noting that $(3(-m)-1)(-m)/2=(3m+1)m/2$ and that the constant term of the product is $1=q^0$, we get $$ \prod_{n=1}^\infty(1-q^n)=\sum_{m=-\infty}^\infty(-1)^m q^{(3m-1)m/2} $$

There are two other proofs you might like: Andrews's proof of Jacobi triple product identity which implies Euler's Pentagonal Theorem, and has a direct bijective proof given by Sylvester (see my survey.

Another, quite different proof is due to Dyson and given here (see my survey and this popular article explaining the connection).