On the vector spaces of Taylor Series and Fourier Series

Solution 1:

First, a general point. The big difference between Taylor and Fourier series is that Taylor series are local and Fourier series are global. That is, Taylor series are defined in terms of and capture local behavior of a function, whereas Fourier series are defined in terms of and capture global behavior. It is not quite accurate to think of a function as being just the sum of its Taylor series; even a function which has a Taylor series need not be equal to it locally (see, for example, MO), and certainly need not have any relationship to it globally, whereas every $L^2$ function on the circle is equal to the sum of its Fourier series in the $L^2$ (not pointwise) sense.

1. Sort of. If you take the inner product to be something like $\int fg \, dx$, then the thing whose inner product with $f$ is $f(0)$ is not quite a function but a distribution, namely the Dirac distribution at $0$. The thing whose inner product with $f$ is $f^{(n)}(0)$, at least on a suitable class of functions $f$, is related to the distributional derivatives of the Dirac distribution.

2. It depends. First, you shouldn't be treating them as bare vector spaces: you really want the language of topological vector spaces and the rest of functional analysis. Second, it depends on where you want the Taylor series to be defined. If we're talking about functions with convergent Taylor series on $\mathbb{R}$ then such functions don't have a Fourier series.

3. The linear algebra analogy is much weaker for Taylor series than it is for Fourier series, the problem being again that Taylor series only capture behavior at a point, and generally functions are determined by their Taylor series to a much weaker extent than functions are determined by their Fourier series. (This is in general; in complex analysis there is a much tighter relationship between Taylor series and Fourier series, as described in the link Martin gives in the comments.)

4. You shouldn't think of Fourier transforms as a generalization of Fourier series. They're both generalized by something called Pontrjagin duality, and I'm not aware of an analogous phenomenon for Taylor series.

5. As mentioned in 2, you might want to learn some functional analysis.