Which role does the $\frac{1}{24}$ in the Dedekind $\eta$-function play?
Solution 1:
The correct but unsatisfying answer is that, given $(q;q)_\infty=\prod_{n=1}^\infty(1-q^n)$, $1/24$ is the value for which $\eta(z)=q^{1/24}(q;q)_\infty$ satisfies $\eta(-1/z)=\sqrt{-iz}\eta(z)$, which leads to all the nice modular form properties. So really, the question you have is: why, conceptually, is $1/24$ the value that leads to that identity?
If you're familiar with Euler's pentagonal number theorem showing that only the terms with exponent $k(3k-1)/2$ are nonzero, then adding $1/24$ produces $$\frac{3}{2}(k^2-2k/6+1/36)=\frac{3}{2}(k-1/6)^2$$ which is a square, implying the eta function to be a sort of askew theta function and the modular relation follows from Poisson summation.
The silly answer (which I believe can be attributed to John Baez) is that while modular forms are defined on the upper half plane, many of the definitions, such as Eisenstein series, should work equally well for $-z$ and $z$; to do this with the Euler product, just replace $1-q^n$ with $q^{-n/2}-q^{n/2}$, or just multiply by $q$ to the power $$-\frac{1}{2}\sum_{n=1}^{\infty}n=-\frac{\zeta(-1)}{2}=\frac{1}{24}.$$ This use of the Riemann zeta function is not the actual value of the divergent sum, of course.
The value $1/24=-\zeta(-1)/2$ can also be "smuggled in" through its relation to $\zeta(2)=\pi^2/6$ just by looking at the asymptotics of the logarithmic derivative. Given the modular equation $\eta(-1/z)=\sqrt{-iz}\eta(z)$, take the derivative $$\frac{1}{z^2}\eta'(-1/z)=\sqrt{-iz}\eta'(z)-\frac{i}{2\sqrt{-iz}}\eta(z)$$ and, letting $F(z)=\eta'(z)/\eta(z)$ while dividing through by $\eta(-1/z)$, get $$F(-1/z)=z^2 F(z) + \frac{z}{2}. $$ If all we knew was $\eta(z)=q^c(q;q)_\infty$, then we'd still be able to get $$F(z)=2\pi i\left(c-\sum_{n=1}^\infty \frac{n q^n}{1-q^n} \right) $$ and for any $c$ to have the modular equation hold, $z^2 F(z)$ must approach $2 \pi i c$ as $z \rightarrow 0$. To find this limit, note $$\sum_{n=1}^\infty \frac{n q^n}{1-q^n}=\sum_{m,n=1}^\infty n q^{mn} = \sum_{m=1}^\infty \frac{q^m}{(1-q^m)^2} $$ and so $$c=\frac{1}{2 \pi i}\lim_{z \rightarrow 0} z^2 F(z)=-\lim_{z \rightarrow 0} \sum_{m=1}^\infty \frac{z^2}{(1-e^{2 \pi i m z})^2}=-\sum_{m=1}^\infty \frac{1}{(2\pi i m)^2}=-\frac{\pi^2/6}{-4\pi^2}=\frac{1}{24}. $$
The function $F$ and its quasi-modular relation are actually linked to the "Eisenstein series" $E_2$ but proving the relation from there is more complicated and implicitly involves reordering sums that aren't absolutely convergent.
Solution 2:
One reason is, that the modular discriminant $Δ(z) = \eta (z)^{24}$ is a modular form of weight $12$. The $\mathbb{C}$-vector space of cusps forms of weight $12$ and level $1$ has dimension $1$, i.e., $\dim S_{12}(SL_2(\mathbb{Z}))=1$, and $\eta (z)^{24}$ is a generator. The presence of $24$ here can also be connected to the Leech lattice, which has $24$ dimensions.