clarification on Taylor's Formula

For instance, take $f(x) = (x^3+1)^3 = x^9+3x^6+3x^3+1$ over $\mathbb{Z}_5$, which has characteristic $5$. Then we have no problems with $D^kf$ for $k \le 4$, but we run into problems with $k=5$ since $5!=0$ in $\mathbb{Z}_5$. However, if we ignore this fact temporarily, we have that

$$\begin{align} \frac{D^5f(x)}{5!} &= \frac{9.8.7.6.5x^4 + 6.5.4.3.2.3x}{5!} \\ &= \frac{4.3.2.1.5x^4 + 1.5.4.3.2.3x}{5!} \\ &= \frac{5!x^4 + 5!.3x}{5!} \\ &= x^4+3x \end{align}$$

So we're fine.

So the "proper interpretation" is, for finite fields of prime order $p$, to work over $\mathbb{Z}[X]$ and then take the natural map $\mathbb{Z}[X] \rightarrow \mathbb{F}[X]$. You might run into issues with finite fields of order $p^n$ for $n>1$, but I hope this illustrates what the author of the book was trying to get at.