Is the size of the Galois group always $n$ factorial?

No I don't think this is right. For example the extension $\Bbb{Q}(\sqrt{3},\sqrt{2})$ is a Galois extension of $\Bbb{Q}$ because it is the splitting field of $(x^2 - 3)(x^2-2)$. This polynomial has exactly 4 distinct roots in $\Bbb{Q}(\sqrt{3},\sqrt{2})$ and it can easily be shown that its Galois group is the Klein 4-group $V_4$. But $V_4$ has order 4 that is not equal to $4!=24$.

Now you are thinking that the Galois group has to be $S_4$. Let me tell you why this is not possible. Let us call $\sqrt{3}$ root #1, $\sqrt{2}$ root #2, $-\sqrt{3}$ root #3, $-\sqrt{2}$ root #4. Given a cycle in $S_4$ let that cycle act on the roots simply by permuting the numbers $1,2,3$ and $4$. For example, the cycle $(12)$ exchanges $\sqrt{3}$ and $\sqrt{2}$ and keeps the negative guys fixed. But then this cannot possibly be a valid element of the Galois group because:

Elements of the Galois group must send for example $\sqrt{2}$ to another root of the minimal polynomial of $\sqrt{2}$ over $\Bbb{Q}$. The minimal polynomial of $\sqrt{2}$ when viewed as an element of $\Bbb{Q}(\sqrt{2},\sqrt{3})$ is $x^2 -2$. Therefore the only possibility for where $\sqrt{2}$ can be sent to is $-\sqrt{2}$, because this is the only other root of this polynomial in any splitting field. So therefore the cycle $(12)$ above cannot be a valid element of the Galois group (if we view the Galois group as sitting inside of $S_4$).

It follows that the Galois group in this case can only be viewed as a proper subgroup of $S_4$ and hence cannot have order $4!=24$.

Edit: Since you seem to be having some trouble understanding the Galois group, let me explain a bit more here. Now I assume that you know what an $F$ - algebra is (otherwise how would you understand field extensions?)

The following is the start of how one describes the Galois group:

Let $A = F[x]$ where $F$ is a field. Let $\iota_A$ denote inclusion of $F$ into $A$. Then any $F$ - algebra homomorphism from $F[x]$ to some other $F$ - algebra $B$ (where the homomorphism in question for $B$ is just the inclusion map $\iota_B : F \to B$) is completely determined by specifying the image of $x$ in $B$. This is because if we have an $F$ - algebra homomorphism $\varphi$ from $F[x] \rightarrow B$, we must have that

$$\iota_B = \varphi \circ \iota_A.$$

In particular this means that $\varphi$ must be the identity on the coefficients of a polynomial in $F[x]$. This explains why $\varphi$ is completely determined by its action on $x$. Now we claim that we have a bijection of sets

$$\Big\{\operatorname{Hom}_{\text{$F$ -algebra}} \big(F[x],B\big)\Big\} \longleftrightarrow B $$

where the bijection is given by $f$ that maps $\varphi$ on the left to $\varphi(x)$ with inverse $g$ that maps an element in $b \in B$ to the homomorphism $\varphi_b$ which is evaluation at $b$. Viz. $\varphi_b$ is just the homomorphism that sends $x$ to $b$. You can check that $f$ and $g$ are mutual inverses.

Now a corollary of this is that we have a bijection

$$\Big\{\operatorname{Hom}_{\text{$F$ -algebra}} \big(F[x]/(f(x)),B\big)\Big\} \longleftrightarrow \Big\{b\in B : \varphi_b(f(x)) = 0 \Big\}. $$

I will leave you to work out the details of how this comes from the fact I stated before. Essentially it is due to the universal property of quotient rings that says given a unique ring homomorphism $\varphi$ from $F[x]$ to $B$ we have unique ring homomorphism from the quotient of $F[x]/(\ker \varphi)$to $B$ such that $\varphi$ factorises through the quotient. I can edit my post to explain this more if you wish.

This is the start of how one gets a description of the Galois group because giving a homomorphism from some field say $F(\alpha)$ to itself (which is automatically an automorphism by the Rank - Nullity Theorem) is by my description above equivalent equivalent to specifying a root of the minimal polynomial of $\alpha$ over $F$ in $B$. But then our $B$ here is exactly what we started with, that is $F(\alpha)$ so that $\alpha$ must be sent to another root of its minimal polynomial over $F$.

Does this help to explain more to you?

Benjamin Lim's answer is great. I thought I would add an answer with infinitely many counterexamples. Let $a\geq 1$ be a natural number, and let $K$ be the splitting field over $\mathbf{Q}$ of $$x^3-ax^2-(a+3)x-1.$$ The discriminant of this polynomial is $(a^2+3a+9)^2$, a square, and the polynomial is irreducible over $\mathbf{Q}$ since it is a cubic without rational roots (the only possibilities for rational roots are $\pm 1$, and neither works). Therefore $\operatorname{Gal}(K/\mathbf{Q})\cong \mathbf{Z}/3\mathbf{Z}$, instead of $S_3$, so the size of the Galois group is $3$, and not $3!=6$. These fields are called the "simplest cubic fields", and you can learn more about them in this article by Daniel Shanks.