Does taking the dot product of two column vectors involve converting one of the vectors into row vectors first?

You're right that there's something going on here.

In a general finite-dimensional vector space, there is no canonical choice of isomorphism from $V$ to $V^*$, even though they're isomorphic because they have the same dimension. However, in a general finite-dimensional vector space, there is also no canonical choice of inner product!

Having an inner product gives us an isomorphism $\phi: V \to V^*$: map a vector $v \in V$ to the element $w \mapsto \langle v,w\rangle$ in $V^*$, and we can check that this will be an isomorphism.

Going the other way is a bit trickier, since inner products need to satisfy $\langle v,v \rangle \ge 0$, but isomorphisms "don't know" about this structure. (In particular, for vector spaces over finite fields, we can have an isomorphism $\phi : V \to V^*$, but it doesn't make sense to have an inner product.) However, if you have an isomorphism $\phi : V \to V^*$, then you can define $\langle v, w\rangle = \phi(v)(w)$, and this will at least be a bilinear form. (Making it artificially symmetric is easy and left as an exercise.)

When we're talking about vectors written as column vectors, we've actually given our vector space lots of structure: we've picked a standard basis, and we're writing our vectors in terms of their coordinates in that basis. Here, taking the transpose to turn a column vector into a row vector is exactly the isomorphism that corresponds to taking the dot product as our inner product.


This sort of question is why I object to using the term “dot product” as a synonym for the Euclidean inner product on $\mathbb R^n$, which is what I assume that you mean here. It’s important to distinguish the two, just as it’s important to distinguish between the elements $v$ of a vector space and their coordinates $[v]_{\mathcal B}$ relative to some ordered basis $\mathcal B$, especially when the vectors themselves are coordinate tuples.

Inner products are independent of the choice of basis. They are just functions that take a pair of vectors and spit out a scalar that have certain nice properties. However, the expression for an inner product $\langle\cdot,\cdot\rangle$ in terms of coordinates of those vectors is basis-dependent. Generally speaking, it’s not the case that $\langle v,w\rangle = [v]_{\mathcal B}^T[w]_{\mathcal B}$. In fact, this only holds when $\mathcal B$ is orthonormal relative to $\langle\cdot,\cdot\rangle$. The right-hand side of that expression is what I would call a “dot product:” it’s a specific computation involving a pair of $n\times 1$ matrices. Now, it happens that if the vectors are elements of $\mathbb R^n$ then the standard basis is orthonormal relative to the Euclidean inner product and their standard coordinate tuples are identical to the vectors themselves, so one can be somewhat cavalier about these distinctions in that context. In general, though, the coordinate formula for an inner product is going to be of the form $\langle v,w\rangle = [v]_{\mathcal B}^TG[w]_{\mathcal B}$ for some fixed symmetric matrix $G$ that’s determined by the inner product and $\mathcal B$. It’s a worthwhile exercise to work out what $G$ is in terms of change-of-basis matrices.

This holds no less in a subspace $V$ of an inner product space. The inner product is inherited from the parent space, and gives the same result regardless of whether we restrict our attention to $V$ or not. On the other hand, its expression in coordinates relative to some basis of $V$ again depends on the choice of basis: if the basis is orthonormal, then it’ll be equal to the dot product of the coordinate tuples, although those coordinate tuples will now be shorter than they were when considering the entire parent space.

There’s a similar condition for turning the application of a covector to a vector into a simple matrix multiplication: If $v\in V$ and $\phi\in V^*$ and we represent the coordinates of a covector as a $1\times n$ matrix, then $\phi(v)=[\phi]_{\mathcal B^*}[v]_{\mathcal B}$ iff the two bases are dual. That is, if $\mathcal B=(v_1,\dots,v_n)$ and $\mathcal B^*=(\beta_1,\dots,\beta_n)$, we must have $\phi_i(v_j)=\delta_{ij}$ for the above identity to hold. To bring this full circle, the Riesz representation theorem connects covectors and inner products: If $H$ is a Hilbert space, then for every element $\phi\in H^*$ there is some fixed $x\in H$ such that $\phi(y)=\langle y,x\rangle$ for all $y\in H$.


I should note that if we’re talking about complex vector spaces, then we need to take the conjugate transpose instead of a simple transpose, i.e., $[v]_{\mathcal B}^H[w]_{\mathcal B}$ instead of $[v]_{\mathcal B}^T[w]_{\mathcal B}$.


No. There is really nothing sacrosanct about a column vector as opposed to a row vector, for one thing.

For another, the dot product of two vectors is defined in a certain way, that makes sense. Namely $\vec a\cdot\vec b=a_1b_1+a_2b_2+\dots+a_nb_n$, where the $a_i$ and $b_i$ are the components of $a$ and $b$ respectively.

As long as you stick to the definition you won't run into trouble.


No. You're confusing the dot product with matrix multiplication.

With the dot product you take two vectors and your final answer is one scalar (number) and the two vectors need to be of the same dimension because that's how the dot product was defined.

For matrix multiplication, you take two matrices and your final answer is another matrix (or a row vector (1xn matrix) or a column vector (nx1 matrix)), but for this you need the number of columns of your first matrix to equal the number of rows of your second matrix because that's how matrix multiplication was defined.


The "dot product" of two vectors $v_{a}$ and $v_{b}$ is often expressed as a matrix multiplication, $$ v_{a} \cdot v_{b} = v_{a}^{T} v_{b}, $$ but can be written without matrix notation as the sum of the pairwise products of the vector components, $$ v_{a} \cdot v_{b} = \sum_{i} v_{a}^{i} v_{b}^{i}. $$

Similarly the matrix-weighted inner product is often expressed via matrix multiplication as $$ \langle v_{a} , v_{b}\rangle = v_{a}^{T} M v_{b}, $$ but can be written without matrix notation as $$ \langle v_{a} , v_{b}\rangle = \sum_{i,j} v_{a}^{i} M_{ij}\, v_{b}^{j}. $$

For both products, transposing $v_{a}$ and using the matrix operation is an implementation of the sum-over-index approach, but doesn't fundamentally change anything about the type of vector that you're working with.