Normalizer of upper triangular group in ${\rm GL}(n,F)$

Solution 1:

$\DeclareMathOperator{\GL}{GL}$$\newcommand{\Span}[1]{\left\langle #1 \right\rangle}$$\newcommand{\Set}[1]{\left\{ #1 \right\}}$Let $e_0, e_1, \dots, e_{n-1}$ be a basis with respect to which $B$ is upper-triangular, and write $$ V_i = \Span{ e_j : j \ge i}. $$ Allow me to use row vectors, so that the group $G = \GL(n, F)$ acts on the right.

Then $$ B = \Set{b \in G : V_i b \subseteq V_i \text{ for each $i$}}. $$ Moreover,

$V_{n-i}$ is the unique subspace $W$ of dimension $i$ such that $W B \subseteq W$.

This is proved by induction on $i$. For $i = 1$ we have that $V_{n-1}$ is the unique common eigenspace for the elements of $B$ (just consider the element of $B$ which is a single Jordan block of size $n \times n$ and eigenvalue $1$, say), then pass to $V / V_{n-1}$ and use induction.

Let $g \in N_{G}(B)$. Then for each $b \in B$ we have $g b g^{-1} \in B$, that is for all $i$ $$ V_{i} g b g^{-1} \subseteq V_i $$ or $$ (V_{i} g) b \subseteq V_i g. $$ It follows from the above that $V_{i} g = V_{i}$, so that $g \in B$.

Solution 2:

It can be shown by direct computation. First, we need the following lemma.

Lemma. Let $g\in GL(n,F)$. Let $h:=g^{-1}$. If there are entries of $g$ and $h$ such that $$g_{ij}\neq 0,h_{kj}\neq 0,i>j,k\geq j,$$ then $g$ is not in $N(B)$, the normalizer of $B$.

(Proof)Let $b=I+E_{jk}$, where $E_{jk}$ is the matrix whose entries are all zero except for the $(j,k)$-entry. Since $k\geq j$, $b$ is an element of $B$. Notice that $(gbg^{-1})_{ij}=(g)_{ij}(h)_{kj}\neq 0$. So $gbg^{-1}$ is not upper triangular(, for $i>j$). Hence $gbg^{-1}\not \in B$. Therefore $g\not \in N(B)$. $\square$

Now we prove that $N(B)=B$ by induction on $n$.

Let us denote by $B_n$ the set of invertible upper triangular matrices of size n and by $N(B_n)$ its normalizer in $GL(n,F)$. Assuming that $N(B_{n-1})=B_{n-1}$, we prove $N(B_{n})=B_{n}$.

Suppose that $g_0\in N(B)$. Let $h_0:=g_0^{-1}$. Since $g_0$ and $h_0$ are invertible, neither of them has a column of zero. If we have $(g_0)_{i,1}\neq 0, (h_0)_{k,1}\neq 0$ for some $i,k$, then the above lemma implies that $i=1$. On the other hand, Since $g_0$ and $h_0$ are invertible, neither of them has a column of zero. Hence $g_0$ has the form$$g_0=\left[\begin{array}{cccc} d_1 & \ast & \cdots & \ast\\ 0 & \ast & \cdots & \ast\\ \vdots & & \vdots\\ 0 & \ast & \cdots & \ast \end{array}\right].$$ We then multiply a product $c_1$ of elementary matrices that corresponds to adding multiples of the first column to other columns, which is in $B$, to obtain an element $g_1:=g_0c_1$ of the form $$g_{1}=\left[\begin{array}{cccc} d_1 & 0 & \cdots & 0\\ 0 & \ast & \cdots & \ast\\ \vdots & & \vdots\\ 0 & \ast & \cdots & \ast \end{array}\right].$$

The $(n-1)\times (n-1)$ matrix $g_1'$ that is obtained by deleting the first row and the first column of $g_1$ is in $N(B_{n-1})$; thus by induction hypothesis, $g_1'\in B_{n-1}$. It follows that $g_1\in B_{n}$. Hence $N(B_{n})\subset B_{n}$.The converse inclusion holds trivially.