Define second derivative ($f''$) without using first derivative ($f'$)

The question I'd like to ask is this:

If $f''(0)$ exists, does $f'$ exist in a neighborhood of $0$?

Of course, under the standard definition of $f''(0)$, we have already assumed that $f'$ exists in a neighborhood of $0$. So instead:

Question: Is there a standard way to define $f''(0)$ as a limit expression that does not include $f'$ in it, and if so, can we deduce from the fact that $f''(0)$ exists that $f'$ exists in a neighborhood of $0$?

Details

If I know what $f'(0)$ is, I can make $f''(0)$ be the constant (if it exists) such that $$ \lim\limits_{h \to 0} \frac{f(h) - \left[ f(0) + f'(0) h + \frac{1}{2} f''(0) h^2\right]}{h^2} = 0. $$ i.e., the Taylor polynomial approximates $f$ to second order. Then, I could just plug in for $f'(0)$ the expression $\frac{f(h) - f(0)}{h}$. But this doesn't work; everything cancels out. Is there a different standard way to define $f''$ without using $f'$?

I should probably put some more work into answering this myself, but first I wanted to see if this is a standard or well-known question.

To the question in the title the answer is Yes. One answer is often used in the theory of numerical differentiation, specifically in finite difference methods. For example:

$$ f''(x) = \lim_{h\to 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2} $$

In response to the very interesting comments: Indeed, the problem with this centered definition is that it has the same problem with defining $f'(x)$ using the expression $$f'(x)=\lim_{h\to 0} \frac{f(x+h)-f(x-h)}{2h}$$ as for any odd function this limit will be zero even if the function is not continuous. Here is another try:

$$f''(x) = \lim_{h\to 0} \frac{f(x)-2f(x+h)+f(x+2h)}{h^2}$$

I wonder if the following formula can solve this issue. In the literature of finite differences, these are known as forward difference and the former formula as a centered difference. A similar "backward" expression can be give, and of course, one expects them to be the same.

About the other question: it might be possible to prove that $f'(0)$ exists when the set of points of discontinuity of $f''$ is of measure zero in an interval containing $0$, since this characterizes Riemann-integrable functions.

This is considered in section 20 of Spivak's Calculus book. The definition the OP proposes for $f''(0)$ fails because, as they already suggest in the question, we won't be able to show that $f'$ exists in a neighborhood of $0$.

The example Spivak gives is $$f(x)=\begin{cases} 0 & x\in\mathbb{Q}\\ x^3 & x\in\mathbb{R}\setminus\mathbb{Q}\end{cases},$$ and the polynomal $p(x)=0$. Then, since $$\lim_{x\rightarrow 0}\frac{f(x)-p(x)}{x^2}=0,$$the OP's definition would imply that $f''(0)=0$. This is however not true since $f$ is not differentiable anywhere but at $0$.