What is the purpose of defining a Hilbert Space?

I understand that the Hilbert space is an infinite dimensional analogue of Euclidean space. However, one thing I have thought about for a while is why certain problems define a variable living within a Hilbert space.

In other words, why do we need to formally define a vector space before working with certain problems? Why can we not just directly do calculations without having to first formally define it?

It seems to me often times that problems involving vectors would be defined as living in a vector space, as a formality, before the real math occurs. Why do we constantly need to mention vector spaces?

The issue is that there are many different infinite-dimensional vector space analogues, and not all of them have nice properties.

Consider $C^\infty(\mathbb R)$, the space of infinitely-differentiable functions on $\mathbb R$. We have that if $f,g$ are infinitely differentiable functions, then so is $f+g$, and for any $\lambda\in\mathbb R$ we have that $\lambda f$ is as well infinitely differentiable.

So, as we don't care the nature of the space we're working in, let's just do some calculations right? What's the inner product on this space? Is there one? Is there even a norm on this space?

The answer to both of those turns out to be no, but without saying "As $C^\infty(\mathbb R)$ is a Frechet space", you may have to do a non-trivial amount of work to prove it. If you fail to do the work, you may run into issues such as: $$\langle f,g\rangle = \int_{\mathbb R}f(x)\overline{g(x)}dx$$ being very infinite for certain values. You may recall that inner product spaces (which Hilbert spaces are) are a subset of normed spaces, as $|\cdot| = \sqrt{\langle\cdot,\cdot\rangle}$ always defines a norm. So, can we put a norm on $C^\infty(\mathbb R)$? Again, no (this time it's again because it's a Frechet Space, Banach Spaces have norms).

So, without ensuring you're in a nice space beforehand, you can get results that really throw off some of your computations.

Additionally, Hilbert Spaces have a property called the Minimum Principle. If $H$ is a Hilbert Space, and $E$ is a "nice" subset of it (meaning closed, convex, and non-empty), then $E$ has an element of minimum norm. This is not true in a generic function space. Famously, while working in PDE's Riemann assumed this was true in a non-Hilbert space, until it was shown by Weierstrauss proved a counterexample (see this history section).

So, we say we work in Hilbert space because not all spaces are created equal, and when we want them to have certain nice properties we have to specify that we're assuming that.

There are many examples of Hilbert spaces that one might not immediately think of as vector spaces. Many famous examples are actually spaces of functions.

For example, $L^2[0,1]$, the space of square-integrable functions on $[0,1]$, is a Hilbert space. The elements of $L^2[0,1]$ are functions; they are not vectors in the "high-school" sense. Nonetheless, you can add them, and you can multiply them by scalars, just like you can do with ordinary vectors.

In fact, $L^2[0,1]$ has something else in common with more familiar spaces of vectors. It has a "basis": the functions $\exp(2\pi n i x)$ with $n \in \mathbb N$. Indeed, every $f \in L^2[0,1]$ is equal (in a certain sense) to a sum of these exponentials of the form $$ f = \sum_{n \in \mathbb N} c_n \exp (2\pi n i x).$$ Perhaps you recognise this kind of expansion: it is the Fourier series of $f$.

Moreover, there is a notion of a "dot product": $$ \langle f , g \rangle = \int_0^1 f^\star g = \sum_{n \in \mathbb N} c_n^\star d_n,$$ where $f = \sum_{n \in \mathbb N} c_n \exp (2\pi n i x)$ and $g = \sum_{n \in \mathbb N} d_n \exp (2\pi n i x)$. If you are indeed a physicist as Matt Samuel claims, you will recognise this as the pairing between "bra"s and "ket"s in quantum mechanics.

So why should you define the formal notion of a vector space, or of a Hilbert space? It is to enable you to treat many different kinds of spaces on the same footing. Once you have checked that a certain space satisfies the definition of a Hilbert space, you are guaranteed that all the general theorems about Hilbert spaces apply to the space you are dealing with, whether the objects in your space are genuine vectors, or whether they are functions, or something else entirely.

For example, the equation I wrote down, $$ \langle f , g \rangle = \sum_{n \in \mathbb N} c_n^\star d_n ,$$ looks very similar to the dot product of two vectors in $\mathbb C^3$, $$\vec{v}.\vec{w} = \sum_i v_i^\star w_i$$ One can prove that a formula of this kind holds for any (separable) Hilbert space, regardless of context.

As people have mentioned in the comments and in the other answer, the process of checking that a given space really does satisfy the definition of a Hilbert space is both important and non-trivial. Not every infinite dimensional inner-product space is a Hilbert space, and if you try to apply Hilbert space theorems to non-Hilbert spaces, you can get in trouble.