Fundamental domain of $\operatorname{GL}(n,\mathbb R)$ acted on by $\operatorname{GL}(n, \mathbb Z)$

Okay, I'm working on some old recollections here of how to do things, so take it with a grain of salt.

First off, we tend to mod out by the maximal compact subgroup on the other side (so we get the symmetric space). So really you want to be working with something like $\mathbb{SO}(n, \mathbb{R}) \backslash \mathbb{SL(n,\mathbb{R})} / \mathbb{SL}(n,\mathbb{Z})$. Except there's an overall scaling factor of $\mathbb{R}^* / \mathbb{Q}^*$ since $\mathbb{GL}(\mathbb{Z})$ has rational determinants, while $\mathbb{GL}(\mathbb{R})$ doesn't.

In the case $n=2$, you can think of this as rotating and rescaling a fundamental parallelogram of the lattice so that one vector is $\hat{x}$, while the second vector lies in a fundamental domain of $H / \mathbb{SL}(2,\mathbb{Z})$.

In the case of higher $n$, you're going to do the same thing -- rotate so that the first vector is $e_1$, the second vector lies in the $(e_1, e_2)$ plane, etc. So think of it as an upper-triangular matrix with positive entries along the diagonal and a 1 in the upper-left corner. You can furthermore arrange things so that each row vector is longer than the one below it ($|\vec{v_i}| \geq |\vec{v_j}|, i<j$). Now, further constraints from the $\mathbb{SL}(\mathbb{Z})$ factor: You can either fix $|a_{ij}| \leq |a_{jj}|/2$ or you can fix $|\vec{v_i}| <= |\vec{v_i} + n * \vec{v_j}| \Leftrightarrow \vec{v_i} \cdot \vec{v_j} \leq |\vec{v_j}|^2$, for $i < j$.

That takes care of (some of) the upper-triangular portion of $\mathbb{SL}(n, \mathbb{Z})$, as well as the $S_n$ subgroup. The rest of it is much trickier, specifically the generators which act like $z \rightarrow -1/z$ in the $\mathbb{SL}(2, \mathbb{Z})$ context.