Differential equations that are also functional

This sort of thing is known as a delay differential equation, and there are lots and lots of books on this subject. They're called "delay" because the constant $\alpha$ is usually chosen to be negative, so that $f'(x)$ depends on the values of $f$ at earlier times.

The equation you have given has an infinite dimensional space of solutions. In particular, if $g\colon [0,\alpha] \to \mathbb{R}$ is any smooth function satisfying $g^{(n)}(\alpha) = g^{(n+1)}(0)$ for all $n \geq 0$, then there exists a unique extension of $g$ to a smooth solution $f$ defined on the entire real line. For $x \geq 0$, this solution is defined by $$ f(x) \;=\; g^{(n)}(x-n\alpha)\qquad\text{where }n = \lfloor x/\alpha\rfloor. $$ while for $x\leq 0$ it involves iterated integrals of $g$.

Jim Belk's answer gives a good explanation of the general context, but I thought I could add an unexpected example of where an equation like this comes up in analytic number theory.

(A small digression, but it is interesting!)

Specifically, a differential delay equation is at the heart of the theory dealing with integers without large prime factors.

Integers without large prime factors:

Let $\psi(x,y)$ denote the number of integers $n\leq x$, all of whose prime factors are $\leq y$. Example: $\psi(20,4)= 10$ because we have the numbers $1,2,3,4,6,8,9,12,16,18$.

Let $y=x^{\frac{1}{u}}$. Then we can ask the question, what does $\psi\left(x, x^{\frac{1}{u}}\right)$ look like as a function of $u$? We get a surprising result:

$$\psi(x, x^{\frac{1}{u}})=\rho(u)x+O\left(\frac{x}{\log x}\right)$$ where $\rho(u)$ satisfies the differential delay equation $$u \rho' (u)=\rho(u-1)$$ with the initial conditions $\rho(u)=1$ when $0\leq u\leq 1$. This is known as the Dickman De-Bruijn function.

There are some delicate details of what range of $u$ does the above hold, and how fast can we let $u$ go to infinity with $x$ and still have $\psi\left(x,x^{\frac{1}{u}}\right)\sim x\rho(u)$. It turns out, such questions are related to the error in the prime number theorem, and we can show that (Hildebrand 1984) the asymptotic holding uniformly in certain ranges of $u$ is equivalent to the Riemann Hypothesis. For a survey of current results regarding $\psi(x,y)$, see Tenenbaum and Hildebrand's 1993 paper: Integers without large prime factors.

(Side note: I posted an answer here about the Maier Matrix method. I just thought I would add, that the Dickman-De-Bruijn function plays an important role in what Maier did.)

Note that $f(x) = e^{\lambda x}$ is a solution of $f'(x) = f(x + \alpha)$ if $\lambda = e^{\lambda \alpha}$, and this equation is satisfied by infinitely many complex $\lambda$ if $\alpha \ne 0$, namely $\lambda = -w_n/\alpha$ where $w_n$ are the branches of ${\rm LambertW}(-\alpha)$.