Do we gain anything interesting if the stabilizer subgroup of a point is normal?

What normality of the stabiliser says is exactly that every group element $g\in G$ that fixes $s$ also fixes the entire orbit $Gs$ pointwise. Conversely any $g\in G$ that fixes any element of the orbit $Gs$ will also fix $s$.

These two parts are equivalent, although the first sentence says that every conjugate of $G_s$ contains $G_s$, while the second sentence says that any conjugate of $G_s$ is contained in $G_s$. Even though a subgroup $H$ may strictly contain a conjugate ${}^gH$ of itself, if it contains all its conjugates then it must be equal to them all, in other words normal (should $H\supsetneq {}^gH$ then ${}^{g^{-1}}H\supsetneq H$, and the hypothesis excludes this).

Observing what happens when a stabilizer is a normal subgroup is one of my favorite pieces of math.

A priori, the conjugation action and normal subgroups do not seem all that important especially when you first begin studying group theory. However, then you are introduced to quotient groups, the Sylow Theorems, etc., and you see that they are indeed important. But the motivation is often lacking, and it seems like happy coincidence that so much information can be derived from these actions and subgroups.

However I believe some light is shed upon conjugation and normal subgroups when you consider group actions, stabilizers, and orbits (which are essentially the most important concepts unifying everything in group theory). The first result that hints that conjugation could be important is that if you have two elements that are in the same orbit of some group action then their stabilizers are conjugate subgroups. This result has already been mentioned.

Once you've recognized conjugation as an action, and you've named the fixed points of this action 'normal subgroups', you can consider what happens when a stabilizer happens to be a normal subgroup. Since by definition, conjugation doesn't move a normal subgroup, if you have the stabilizer of some element which is normal then all elements in the same orbit have the exact same stabilizer. In particular, this means that the only elements of the group which actually move the elements in the orbit are the cosets of the normal stabilizer. More interestingly, if you have two group elements from the same coset of that normal stabilizer, then they act on the orbit in the exact same manner.

These observations can actually motivate the construction of the quotient group. Let $G$ be a group, and let $N$ be a normal subgroup. Consider the coset space $G/N$, and let $G$ act on $G/N$ by left multiplication. We know in general that if we let $G$ act on $G/H$ for some subgroup $H$ by left multiplication, then the kernel of the action is $H$ and the action is transitive. However we get very convenient facts when we consider a normal subgroup. Namely, the stabilizer of any one coset $aN$ is $N$. This implies that $N\cdot aN=aN$ (as sets). Further since the cosets of $N$ act on $G/N$ in the exact same manner, we get that for all $b, b'\in bN$ that $b\cdot aN=b'\cdot aN$. This implies that for any two cosets $aN$ and $bN$ in $G/N$ that $aN\cdot bN=(ab)N$ (as sets). This means that the product set of any two cosets of $N$ is again a coset of $N$. This is a nontrivial relation. In fact, if $H$ is a subgroup of $G$ such that the product set of any two cosets of $H$ is also a coset of $H$, then $H$ must be normal (this result has been discussed here before). So, this property characterizes normal subgroups. These facts also allow us to see that $\cdot$ (subset product) is a group operation on $G/N$ whereas normally it's not even a binary operator on $G/H$ for a non-normal subgroup $H$.

The stabilizer of a point $s \in S$ is normal in $G$ if and only if every element that stabilizes $s$ stabilizes the orbit $G\cdot s$ of $s$ pointwise.

By Wielandt 1964, p.13,

Proposition 7.1 If the transitive group G contains an intransitive normal subgroup N different from 1, then [...] the orbits of G form a complete block system of G.

Being intransitive G has to stabilize something smaller than S itself. If G is a point stabilizer, then {a} is an orbit for G and so will be all elements of S.

Reversely (proposition 7.2), given a block system, the elements of G permute the blocks via a natural homomorphism. The kernel of this homomorphism stabilize each block - and it is a normal subgroup.

eg: the corners of the Rubik cube are blocks (we disregard the edges). Then the kernel contains exactly those permutations that spin the corners without displacing them.

:) if we scramble the corners, spin, than scramble back, we got a spin.

For a transitive action, a point stabilizer is normal if and only if it equals the kernel of the action, or said differently if and only if it fixes every point. That said, a point stabilizer is normal iff it equals the kernel of the group action restricted to the orbit containing the point stabilized. Hence for an abelian or cyclic group acting transitive the notion of kernel and stabilizer are the same.