Are $3$ and $11$ the only common prime factors in $\sum\limits_{k=1}^N k!$ for $N\geq 10$?

As pointed out in the comments, the case is trivial( at least if you know some theorems) if you fix $n>10$, instead of $n>p-1$ for prime $p$.

It follows from Wilson's Theorem, that if you have a multiple of $p$ at index $n= p-2$ you won't at index $p-1$ because it will decrease out of being one for that index. $p>12$ implies at least $1$ index where it is NOT a multiple if it worked at index $11$. That leaves us with $p<12$ which would have to be factors of the sum up to $p-1$, $2$ is out as the sum is odd, $5$, needs $24+1+2+6=33$ to be a multiple of $5$, it isn't. Lastly $7$ needs $1+2+6+24+120+720=873$ to be a multiple which would force $33$ to be a multiple of $7$ which it isn't.