Why consider square-integrable functions?
Solution 1:
The reason that spaces of square integrable functions arose in the first place was to study the orthogonal trigonometeric (Fourier) series. Interestingly, Parseval had already noted in 1799 the equality that now bears his name: $$ \frac{1}{\pi}\int_{-\pi}^{\pi}f(x)^{2}\,dx = \frac{1}{2}a_{0}^{2}+\sum_{n=1}^{\infty}a_{n}^{2}+b_{n}^{2}, $$ where $a_{n}$, $b_{n}$ are the (Fourier) coefficients $$ a_{n}=\int_{-\pi}^{\pi}f(x)\cos(nx)\,dx,\;\;\; b_{n}=\int_{-\pi}^{\pi}f(x)\sin(nx)\,dx. $$ This comes out of the orthogonality conditions for the $\sin(nx)$, $\cos(nx)$ terms in the Fourier series. No definite connection was seen between Euclidean N-space and the above at that time; such a connection took decades to evolve. But square-integrable functions gained interest in the early 19th century, and especially after the early 19th century work of Fourier.
It took some time to see a general Cauchy-Schwarz inequality, and to begin to see a connection with geometry, eventually leading to inner-product space abstraction for the space of square-integrable functions. The CS inequality wasn't widely known until after the 1883 publication of Schwarz, even though essentially the same result was published in 1859 by another author. Hilbert proposed his $l^{2}$ space by the early 20th centry as an abstraction of the square-summable Fourier coefficient space, but also a abstraction of finite-dimensional Euclidean space. The connection with square-integrable functions was already firmly established.
In hindsight we can see good reasons that square-integrable functions are connected with energy, and other Physics concepts, but the abstraction seems to have been dictated more out of solving equations using 'orthogonality' conditions. Of course many of the equations arose out of solving physical problems; so it's also hard to separate the two. Now, after the fact, there is interpretation of the integral of the square of a function. On the other hand, the Mathematical abstraction of dealing with functions as points in a space, with distance and geometry on those points has been even more far-reaching, and a great part of the impetus for modern abstract and rigorous Mathematics.
Note: All of this happened before Quantum Mechanics.
Reference: J. Dieudonne, "History of Functional Analysis".
Solution 2:
There are several reasons, you named some. You need to work with the space to get a feeling why these things are important.
One reason you did not mention which makes them very popular is the fact that they are very well suited to study elliptic partial differential equations, esp of second order. By partial integration, assuming zero boundary conditions, the Laplacian is intimately connected to the square integral of the gradient
$$ \int \Delta u = \int \langle\nabla u, \nabla u\rangle $$
This makes the integral on the right an important object of study and, naturally, you will look at those functions for which this is finite.
Since the Laplacian is the model operator for elliptic second order differential equations (which may actually be true because it is related to the term in the above equation, which, in a sense, behaves like an energy term) this is one of the very fundamental reasons why these spaces are so important.
In addition, there is a very fundamental isometry of $L^2$, the Fourier transform. This is also intimately connected to the study of elliptic PDE. As an isometry it behaves particularly well on $L^2$ and allows to use the full power of Hilbert space theory to be applied to the study of elliptic PDE.