Applications of Gauss sums

Gauss sums are not an ad-hoc construction! I know two ways to motivate the definition, one of which requires that you know a little Galois theory and the other which is totally mysterious to me.

Here is the Galois-theoretic explanation. Let $\zeta_p$ be a primitive $p^{th}$ root of unity, for $p$ prime. The cyclotomic field $\mathbb{Q}(\zeta_p)$ is Galois, so one can define its Galois group, the group of all field automorphisms which preserve $\mathbb{Q}$. Such an automorphism is determined by what it does to $\zeta_p$, and it must send $\zeta_p$ to another primitive $p^{th}$ root of unity. It follows that the Galois group $G = \text{Gal}(\mathbb{Q}(\zeta_p)/\mathbb{Q})$ is isomorphic to $(\mathbb{Z}/p\mathbb{Z})^{\times}$, which is cyclic of order $p-1$.

Now suppose $p$ is odd. As a cyclic group of even order, $G$ has a unique subgroup $H$ of index two given precisely by the multiplicative group of quadratic residues $\bmod p$, so by the fundamental theorem of Galois theory the fixed field $\mathbb{Q}(\zeta_p)^H$ is the unique quadratic subextension of $\mathbb{Q}(\zeta_p)$. And it's not hard to see that this unique quadratic subextension must be generated by

$$\sum_{\sigma \in H} \sigma(\zeta_p) = \sum_{a \text{ is a QR}} \zeta_p^a = \frac{1}{2} \left( \sum_{a=1}^{p-1} \zeta_p^{a^2} \right)$$

which you will of course recognize as a Gauss sum! So the Gauss sum generates a quadratic subextension, and any of various methods will tell you that this subextension is precisely $\mathbb{Q}(\sqrt{p^{\ast}})$ where $p^{\ast} = (-1)^{ \frac{p-1}{2} } p$. (This does not actually require any computation: if you know enough algebraic number theory, it follows from a consideration of which primes ramify in cyclotomic extensions.)

The totally mysterious explanation is that Gauss sums naturally appear when you start thinking about the discrete Fourier transform. For example, the trace of the DFT matrix is a Gauss sum. But more mysteriously, Gauss sums are eigenfunctions of the DFT in a certain sense. (I sketch how this works here.) There is a sort of mysterious connection here to the Gaussian distribution, which is an eigenfunction of the continuous Fourier transform; see this MO question. Again, I don't know what to make of this. There is a book by Berg called The Fourier-analytic proof of quadratic reciprocity and it may or may not be about this construction.

Not just quadratic reciprocity, one can use them to prove higher reciprocity laws: see Ireland and Rosen's A Classical Introduction to Modern Number Theory. They also turn up in the functional equation for Dirichlet L-functions (and are massively generalized in the topic of root numbers).

It can be used to prove a certain theorem called the Stickelberger's theorem, the likes of which can in turn be used for proving things like Catalan's Conjecture.