Purpose of cusps
In the theory of of modular forms, there is the set of of cusps defined by $\mathbb{P}^1 (\mathbb{Q})= \mathbb{Q} \cup \{\infty\}$. For an subgroup $\Gamma < \text{SL}_2(\mathbb{Z})$ of finite index, we call $\Gamma\backslash \mathbb{P}^1 (\mathbb{Q})$ the set of cusp classes.
For example, I know that $\text{SL}_2(\mathbb{Z})/\mathbb{P}^1 (\mathbb{Q})$ consists of one element, i.e. represented by $\infty$; Modular forms which are written as Taylor series and vanish at any belonging cusp are named cusp forms.
However, what is so important of these two definitions? Do they have 'special' properties, i.e. allow a geometric interpretation of modular forms? Which are the features they provide and in which contexts of modular forms I maybe re-encounter them?
Solution 1:
The moduli space of 1-complex dimensional tori (of area 1) is (geometrically) a non-compact orbifold. The non-compactness comes from the fact that on a torus, one may decrease the length of a closed curve down to zero. The universal cover of moduli space is the hyperbolic plane $\mathbb{H}^2$, and the (orbifold) fundamental group of moduli space is $\Gamma=PSL_2(\mathbb{Z})=Mod(T^2)$, which acts on $\partial\mathbb{H}^2=\mathbb{P}^1(\mathbb{R})$ in such a way that it preserves the cusp set $\mathbb{P}^1(\mathbb{Q})$, which parameterize the curves on a torus, corresponding to $\pm(p,q)\in \mathbb{Z}^2, \gcd(p,q)=1$, representing primitive elements of $\pi_1(T^2)=\pi_1(\mathbb{R}^2/\mathbb{Z}^2)=\mathbb{Z}^2$.
Congruence subgroups of $\Gamma$ generally correspond to subgroups which preserve a "level structure" on the torus $T^2$. That is, there is some cover $\tilde{T}^2\to T^2$, and one considers the subgroup of $Mod(T^2)$ which lifts to this cover. More generally, one can intersect subgroups of this sort to obtain congruence subgroups. Under the action of this subgroup of $Mod(T^2)$, the simple closed curves may split up into finitely many orbits, which will correspond to the cusps of the subgroup. For example, if one takes the cover corresponding to the subgroup $\Gamma_0(N)$, corresponding to the subgroup of $PSL_2(\mathbb{Z})$ which preserves the lattice $\mathbb{Z}+N\mathbb{Z} \leq \mathbb{Z}+\mathbb{Z}=\pi_1(T^2)$, then the curves corresponding to $(1,0)=\infty$ and $(0,1)=0\in \mathbb{P}^1(\mathbb{Q})$ will no longer be in the same orbit, so there are at least two cusps.
Now, by the uniformization theorem, for a finite-index subgroup $\tilde{\Gamma}\leq \Gamma$, there is a compact Riemann surface $\overline{X}$ compactifying $X=\mathbb{H}^2/\tilde{\Gamma}$. The holomorphic cusp forms (of even weights) may be interpreted as the forms which descend to the compactified surface $\overline{X}$ as holomorphic differential forms with coefficients in a line bundle. For example, weight 2 cusp forms will correspond to holomorphic 1-forms on $\overline{X}$. The space of these (by the Hodge theorem, or Riemann-Roch) is isomorphic to $H^1(\overline{X},\mathbb{C})\cong \mathbb{C}^g$, where $g$ is the genus of $\overline{X}$. If one did not impose the cusp condition, then $H^1(X)$ would have larger dimension, and there would be further 1-forms not vanishing at the cusps.
In the case of non-holomorphic modular functions, such as Maass wave forms, which correspond to eigenfunctions of the Laplacian on $X$, there are classes of eigenfunctions of the Laplacian (which are not in $L^2(X)$) which come from the cusps, and are represented by Eisenstein series. These forms have a continuous spectrum. Certain arithmetic information though tends to be represented by the cusp forms (which are in $L^2(X)$), which have a discrete spectrum. For example, Selberg's eigenvalue conjecture says that the cusp Maass forms always have eigenvalue $\geq \frac14$ for a congruence subgroup $\tilde{\Gamma}$; the corresponding fact is already known for the Eisenstein eigenfunctions.
Solution 2:
On non-compact geometric objects, even with finite total measure, functions that decay nicely on the "not compact parts" are obviously more tractable analytically. In the specific situation of "holomorphic elliptic modular forms", there is a (curious!) bifurcation between "rapidly decreasing" and "moderately growing", the former (holomorphic) cuspforms and the latter (holomorphic) Eisenstein series.
That situation is so simple in some ways, e.g., finite-dimensionality of holomorphic modular forms of a given weight, that (perhaps mercifully) analytical issues are easily overlooked.
For "waveforms", the analytical issues become more vivid, because (as Selberg demonstrated) the space of cuspforms is infinite-dimensional... and the "continuous spectrum" for the invariant Laplacian on $SL_2(\mathbb Z)\backslash\mathfrak H$ is spanned by Eisenstein series, which are not in $L^2$.
Yet, again, the space of $L^2$ automorphic forms in fact has a basis of rapidly-decreasing functions for the bulk of its discrete decomposition, namely, cuspforms (waveforms), and then a much smaller leftover part.
That is, the notion of "cuspform" (as improved and generalized, modernized, by Gelfand et al) _has_proven_ to be the correct distinction... It is not obvious a-priori, nor does the holomorphic case (dating back well into the 19th century) give clear evidence of the larger story.
Solution 3:
As Paul Garrett intimates in his answer, the notion of cuspform is fundamental, but can be subtle in its meaning and implications. Here are a few different points of view that might help:
From the point of view of someone interested in the relationship between modular forms and algebraic number theory, Hecke eigenforms correspond to (certain) two-dimensional representations of the absolute Galois group $\mathrm{Gal}(\overline{\mathbb Q}/\mathbb Q)$. Under this correspondence, the Eisenstein series correspond to reducible two-dimensional representations (i.e. those which are the sum of two characters), while the cuspidal eigenforms correspond to irreducible representations.
A different answer, related to quadratic forms:
The $\theta$ series of a positive definite quadratic form will is a modular form. If we expand it in terms of a basis of Hecke eigenforms, it will have some Eisenstein contributions, which can be more-or-less computed explicitly, and then some cuspform contributions, which in general aren't describable by an explicit formula.
However, we know that the Hecke eigenvalues, and hence the $q$-expansion coefficients, of a cuspform grow at a much slower rate than those of an Eisenstein series. (There are elementary bounds, due to Hecke, tighter bounds, due to Rankin, and then the ultimate Ramanujan--Petersson bound, proved by Deligne.) Just from this, we can derive asymptotics as $n \to \infty$ for the number of representations of $n$ by a given quadratic forms. (See e.g. Serre's Course in arithemtic for concrete examples.)
In weight two, as Agol notes in his answer, cuspforms correspond to holomorphic differential forms on the modular curve, while all modular forms correspond to differential forms that are holomorphic except for possible simple poles at the cusps.
Just to tie in a little with Paul Garrett's answer, in the spectral theoretic point of view on automorphic forms (not something you see in the beginning literature on holomorphic modular forms that number theory students tend to read, but something that is usually introduced in books that take a more representation-theoretic veiw-point), cuspforms (on a given group, and on its Levi subgroups) are the basic building blocks in terms of which the rest of the spectral theory is developed.