Does swapping columns of a matrix cause the rows of the inverse matrix to be swapped?

This question came up while I was performing some computation on a few matrices on an unrelated problem in computer science.

Let $A$ be an invertible matrix with columns $A_1, \dots A_n$. Let $B$ be its inverse, with rows $B_1, \dots, B_n$. Now construct a new matrix $\hat{A}$ by swapping two columns $A_i$ and $A_j$. Let $\hat{B}$ be the inverse of $\hat{A}$.

In my specific computations, I noticed that $\hat{B}$ was nothing more than $B$ with the rows $B_i$ and $B_j$ swapped. Is this always true? I doubt it was a coincidence because the numbers were quite random.

Note: I was working with real numbers but I'd be interested to know if the field makes any difference.

Yes, this is always true. Note that swapping columns $i$ and $j$ is equivalent to multiplying on the right side by the elementary matrix $T_{ij}$ which is defined by swapping rows $i$ and $j$ of the identity matrix. You can check that this matrix is the inverse of itself. Also, multiplying by $T_{ij}$ on the left side is equivalent to swapping rows $i$ and $j$.

So if we call your matrix $A$ then $AT_{ij}$ is the matrix that you get that swapping columns $i$ and $j$. Then its inverse is $T_{ij}A^{-1}$ which is the matrix that you get when you swap rows $i$ and $j$ in $A^{-1}$.

Another way to think of it: you are just re-numbering the basis for one of the vector spaces: The domain space for $A$, which is the range space fo $A^{-1}$.

Your observation is spot on. You can swap columns of a matrix by right-multiplying by a permutation matrix $P$ that is the identity matrix with the corresponding columns swapped: $\hat A = AP$. We then have $$\hat A^{-1} = (AP)^{-1} = P^{-1}A^{-1}.$$ $P$ is its own inverse, and left-multiplying by $P$ swaps the rows of $A^{-1}$ that are swapped in $P$.