Extension of real analytic function to a complex analytic function
Solution 1:
Yes, a real analytic function on $\Bbb R$ extends locally to a complex analytic function, except that (in my opinion) "locally" doesn't/shouldn't mean what you say it does.
If $f$ is real analytic on $\Bbb R$ then there exists an open set $\Omega\subset\Bbb C$ with $\Bbb R\subset\Omega$, such that $f$ extends to a function complex-analytic in $\Omega$. This is easy to show - details on request. But $f$ need not extend to a set $\Omega$ that contains some strip $\Bbb R\times(-\epsilon,\epsilon)$.
For example consider $$f(t)=\sum_{n=1}^\infty a_n\frac{1}{n^2(n-t)^2+1},$$where $a_n>0$ tends to $0$ fast enough. The extension will have poles at $n+i/n$, so $\Omega$ cannot contain that horizontal strip.
Solution 2:
We indeed have a local extension, given by the Taylor expansion. That is, given $x_0\in\mathbb R$ there exists $\epsilon>0$ such that $$\sum_{n=0}^\infty\frac{f^{(n)}(x_0)}{n!}(z-x_0)^n$$ converges for $|z-x_0|<\epsilon$, and we call the limit $g(z)$. This defines an analytic function $g$ on a neighborhood of $\mathbb R$ which clearly extends $f$. (We do need to verify that if $z$ is in the disk of convergence of two points $x_0$ and $\tilde x_0$ then the series above is equal to the same series with $x_0$ replaced by $\tilde x_0$, but this follows from the identity theorem.)
It is worth pointing out that the $\epsilon>0$ described above depends on $x_0$, and there does not necessarily exist such $\epsilon$ independent of $x_0$. So in general we cannot expect the neighborhood to be of the form $\mathbb R\times(-\epsilon,\epsilon)$.
Solution 3:
As David C. Ullrich and Jason pointed out, the answer is not always affirmative. Let me quote a Theorem that gives a partial affirmative answer to your question, with somewhat different assumptions.
Theorem: Assume that $f$ satisfies the Fourier inversion formula, i.e. $f(x) = \int_{\mathbb{R}}\hat{f}(\xi)e^{2\pi i x\xi}\,d\xi$ and that $$|\hat{f}(\xi)| \le Ae^{-2\pi a|\xi|}$$ for some constants $a,A > 0$. Then $f$ is the restriction to $\mathbb{R}$ of a function $f(z)$ holomorphic in the strip $S_b = \{z \in \mathbb{C}: |\text{Im}(z)| < b\}$, for any $0 < b < a$.
I think this is an interesting result since it also tells how much you can push the $\epsilon$ is you question. You can find a proof of this theorem and further discussions on the possibility of extending functions on the real line to complex-analytic functions in Complex Analysis by E.M. Stein and R. Shakarchi.