Second derivative of $f(f(\cdots f(x)\cdots )?$

Let's take $f^{\prime\prime}_4(x)$ as an example.

We have $f^\prime_4(x)=f^\prime[f(f(f(x)))]\cdot f^\prime[f(f(x))]\cdot f^\prime[f(x)]\cdot f^\prime(x)$, by the chain rule.

The product rule gives $\frac{d}{dx}p(x)\cdot q(x)\cdot r(x)\cdot s(x)=$$p^\prime(x)\cdot q(x)r(x)s(x)+q^\prime(x)\cdot p(x)r(x)s(x)+r^\prime(x)\cdot p(x)q(x)s(x)+s^\prime(x)\cdot p(x)q(x)r(x)$.

So we just calculate the derivative of each term of the product, and evaluate the other terms. All terms evaluate down to $\alpha$, so our example reduces to $\alpha^3\cdot[p^\prime(x)+q^\prime(x)+r^\prime(x)+s^\prime(x)]$.

$p^\prime(x)=f^{\prime\prime}[f(f(f(x)))]\cdot f^\prime[f(f(x))]\cdot f^\prime[f(x)]\cdot f^\prime(x)=\beta\alpha^3$.
$q^\prime(x)=f^{\prime\prime}[f(f(x))]\cdot f^\prime[f(x)]\cdot f^\prime(x)=\beta\alpha^2$.
$r^\prime(x)=f^{\prime\prime}[f(x)]\cdot f^\prime(x)=\beta\alpha$.
$s^\prime(x)=f^{\prime\prime}(x)=\beta$.

The answer is then $\beta\alpha^3(\alpha^3+\alpha^2+\alpha+1)$. This last sum can be written as $\frac{\alpha^4-1}{\alpha-1}$, when $\alpha\neq1$.

Generalizing, we have $\beta\frac{(\alpha^{n-1})(\alpha^n-1)}{\alpha-1}$, or $n\beta$ when $\alpha=1$.

That's not a proof, but it's good enough to spot the pattern.

Now somebody generalize it to $f^{(m)}_n(0)$. ;-)

Note that if $g(x)=ax+bx^2+O(x^3)$ and $h(x)=cx+dx^2+O(x^3)$ then $$\begin{align} g(h(x)) &=a(cx+dx^2+O(x^3))+b(cx+dx^2+O(x^3))^2+O(x^3)\\ &=acx+(ad+bc^2)x^2+O(x^3) \end{align}$$ so if we let $g=f$ and $h=f_n$ we get the recurrence $$f_{n+1}''(0)=\alpha f_n''(0)+\beta f_n'(0)^2=\alpha f_n''(0)+\beta\alpha^{2n}.$$