Why are inner products in RKHS linear evaluation functionals?

Solution 1:

On a Hilbert space, all continuous linear functionals are inner product functionals (Riesz), and conversely (Cauchy-Schwarz). In a RKHS, evaluation functionals are continuous, which is equivalent to being inner product functionals. The converse is usually not true. That is, inner product functionals on a RKHS need not be point evaluations.

Let $H$ be a Hilbert space consisting of complex-valued functions on a set $X$ such that for each $x\in X$, the evaluation functional $f\mapsto f(x)$ is continuous. Then for each $x\in X$ there is a $k_x\in H$ such that for all $f\in H$, $f(x)=\langle f,k_x\rangle$. (The function $K:X\times X\to \mathbb{C}$ defined by $K(x,y)=k_y(x)=\langle k_y,k_x\rangle$ is the reproducing kernel of the RKHS $H$, and some authors start with $K$ in defining a RKHS.) Only the inner products with the elements $k_x$ are evaluation functionals.

For example, consider the Hardy space $H^2$ of holomorphic functions on the open unit disk whose sequences of Maclaurin series coefficients are in $\ell^2$, with inner product $\displaystyle{\left\langle \sum_{k=0}^\infty a_kz^k,\sum_{k=0}^\infty b_kz^k\right\rangle=\sum_{k=0}^\infty a_k\overline{b_k}}$. Evaluations on the open disk are continuous, as can be seen directly by writing down the element $k_w$ of $H^2$ whose inner product functional is evaluation at $w$, $k_w(z)=\sum_{k\geq0} \overline{w}^kz^k=\frac{1}{1-\overline{w}z}$. So a necessary and sufficient condition for an inner product functional $f\mapsto\langle f,g\rangle$ to be an evaluation functional is the existence of a $w$ in the open unit disk such that $g=k_w$, a condition which typical $g\in H^2$ will not satisfy. Note that the set of evaluation functionals is not closed under scalar multiplication, nor addition. In fact, it is linearly independent.

A simpler but in some ways less interesting example is $\ell^2$ thought of as a space of functions on the nonnegative integers, where the evaluation functionals are just the inner products with elements of $\ell^2$ that have the value $1$ at one point and vanish elsewhere. An even simpler example would be a finite dimensional Hilbert space thought of as functions on a finite set. In these cases, cardinality is enough to see that most inner product functionals are not point evaluations.

The inner product functionals and evaluation functionals would be identical if you were considering a Hilbert space $H$ as a space of functions on its dual space, in the usual isomorphism of $H$ with its double dual.

Added: Here is some elaboration on the first 2 sentences. If $H$ is a Hilbert space and $g\in H$, then the function $T_g :H\to \mathbb{C}$ defined by $T_g(f)=\langle f,g\rangle$ is a linear functional on $H$ called an "inner product functional" above. Each inner product functional is continuous. The operator norm of $T_g$ is equal to $\|g\|$. The Cauchy-Schwarz inequality gives $|T_g(f)|\leq \|f\|\|g\|$ for all $f$, which means $\|T_g\|\leq\|g\|$. Plugging $g$ into $T_g$ gives $|T_g(g)|=\|g\|^2$, showing that $\|T_g\|\geq \|g\|$.

So inner product functionals are continuous, and this would be true in any inner product space. The Riesz representation theorem (for Hilbert space, sometimes also called Riesz's lemma) says that the converse is true for a Hilbert space. You can see this for example in the Wikipedia article, and in many textbooks including the basics of Hilbert spaces, such as Rudin's Real and complex analysis. That is, if $T:H\to\mathbb{C}$ is any continuous linear functional, then there is a $g\in H$ such that $T=T_g$.

Hopefully the first sentence is clearer now. As for the second sentence, it follows directly from the first sentence and the definition of RKHS, and the second paragraph elaborates on this. There is more than one way to characterize RKHS, and if continuity of point evaluations isn't clear from your definition, perhaps you could provide the definition to make it easier to answer your questions.