Relationship between inner product and norm

Yes, there are many different types of inner products. Consider the inner product on $L^2$ given by $\langle f, g \rangle = \int f(x) \overline{g(x)} dx$.

An inner product $\langle , \rangle$ always defines a norm by the formula $||x||^2 = \langle x, x \rangle$. You can check that all the conditions of a norm are satisfied. However, the converse is not true, that is, not every norm gives rise to an inner product. Norms which satisfy the parallelogram law can be used to define inner products via the polarization identity.


In $\mathbb{R}^n$ and $\mathbb{C}^n$ we can actually present a characterization of all inner products. Fix an inner product $\langle \cdot , \cdot \rangle$, probably the Euclidean one. Then for any positive definite matrix $P$, $(x,y) = \langle x,Py \rangle$ is also an inner product. Here positive definiteness is defined in terms of $\langle \cdot , \cdot \rangle$.

Conversely, any inner product on a finite dimensional space over $\mathbb{R}$ or $\mathbb{C}$ has such a representation. In particular, given an inner product $(\cdot , \cdot)$, we have $P_{ij}=( e_i,e_j )$, where $\{ e_i \}_{i=1}^n$ is an orthonormal system with respect to $\langle \cdot , \cdot \rangle$. This gives the representation of $P$ in terms of the orthonormal coordinates given by $\langle \cdot , \cdot \rangle$. If the base inner product is the Euclidean one, then $\{ e_i \}_{i=1}^n$ can be taken to be the standard basis, in which case this is the usual representation.

The fact that that every inner product induces a norm is nearly a simple consequence of the definition of an inner product. Specifically, $\| x \| \geq 0$ with $\| x \| = 0$ iff $x = 0$ is built completely into the definition. $\| a x \| = |a| \| x \|$ is also trivial, because $\| a x \| = (ax,ax) = a \overline{a} \| x \|^2$, then $|a| = \sqrt{a \overline{a}}$. The tricky part is the triangle inequality. This is typically proven with the Cauchy-Schwarz inequality.


The weighted sum formula for the inner product, $$(x,y) = \sum_i w_i x_i y_i,$$ captures the inner product's essential meaning - in finite dimensions all inner products act like a weighted sum in some basis (also the idea can be generalized to infinite dimensions using the spectral theorem, the sum basically becoming an integral)

Why? Well by definition inner products are symmetric positive definite bilinear forms, so on finite dimensions they always have a representation by a symmetric positive definite matrix, $$(x,y) = x^T M y.$$

Taking the singular value decomposition $M = U \Sigma U^T$ (or you could do eigenvalue decomposition since it's a SPD matrix), that gives $$(x,y) = x^T U \Sigma U^T y$$

So, if $x_i,y_i$ are the components of $x,y$ in the basis of the singular vectors of $M$ then you can write the inner product in a weighted sum type form $$(x,y) = \sum_i \sigma_i x_i y_i,$$

where the $\sigma_i$'s are the singular values from the diagonal matrix $\Sigma$ .


In terms of norms, the unit balls for a norm induced by an inner product are ellipsoids, with axes given by the singular vectors, and axis lengths determined by the singular values. So in a more geometric sense, there is a direct correspondence between ellipsoids and inner products.


Any pair of inner product spaces of equal and finite dimension are indeed isomorphic, so there is no meaningfully different inner product; to demonstrate that, notice that, from any basis $b_1,b_2,\ldots,b_n$ of a vector space, you can, knowing the inner product, remove the components of $b_i$ parallel to $b_1,\ldots,b_{i-1}$, ensuring that it is orthogonal to any of those, and thereby, it must be possible to find a orthogonal basis of unit vectors. A linear map between two spaces of equal dimension, taking one orthogonal basis to another obviously preserves inner products, thus the spaces are isomorphic.

So, there is no inner product, for instance, inducing the norm $||x||=|x_1|+|x_2|$ on $\mathbb{R^2}$ - so the "circle" of vectors where $||v||=1$ is always just a linear transformation of a sphere when the norm is induced by an inner product - and, for finite spaces, it suffices to show that any inner product induces a norm, since a norm is still a norm after application of a linear map.

The only non-trivial thing necessary to show that the inner product space induces a norm is the triangle inequality. Note that, in any space, if we need $||x+y||\leq||x||+||y||$ for particular vectors $x$ and $y$, we just need that, in the span of $x$ and $y$, this holds; however, since the span of $x$ and $y$ is a (at most) two dimensional linear subspace of the inner product space (which is, in itself, a two-dimension inner-product space, obviously). Since the triangle inequality holds for any two-dimensional inner product space (remember that they are all isomorphic), it holds in every inner product space.