$f^{(3)}(x) h$ was sucked in big O

Solution 1:

Let's start here:

$$ f''(x) + f^{(3)}(x) h + O(h^2).$$

In your context, $x$ is treated as a fixed quantity whereas $h$ is small, so $f^{(3)}(x) h$ is effectively a constant times $h$. A constant times $h$ is just about as pure an $O(h)$ term as you can get.

Writing $f^{(3)}(x) h$ as $O(h)$ we have

$$ f''(x) + O(h) + O(h^2).$$

Now since we're doing big-O of small $h,$ the $O(h)$ term dominates the $O(h^2)$ term. Example: $2h \in O(h)$ and $5h^2 \in O(h^2)$ but $2h + 5h^2 \in O(h).$ In general adding any $O(h^2)$ function to any $O(h)$ function gives you an $O(h)$ function, so we really just have

$$ f''(x) + O(h).$$