Understanding positive definite kernel

From Mercer's Theorem:

A kernel is a symmetric continuous function $ K: [a,b] \times [a,b] \rightarrow \mathbb{R}$, so that $K(x, s) = K(s, x)$ ($\forall s,x \in [a,b]$).

$K$ is said to be non-negative definite (or positive semi-definite) if and only if $$\sum_{i=1}^n\sum_{j=1}^n K(x_i, x_j) c_i c_j \geq 0$$ for all finite sequences of points $x_1, ..., x_n$ of $[a, b]$ and all choices of real numbers $c_1, ..., c_n$.

From Positive-definite kernel:

Let $\{ H_n \}_{n \in {\mathbb Z}}$ be a sequence of (complex) Hilbert spaces and $\mathcal{L}(H_i, H_j)$ be the bounded operators from $H_i$ to $H_j$.

A map $A$ on ${\mathbb Z} \times {\mathbb Z}$ where $A(i,j)\in\mathcal{L}(H_i, H_j)$ is called a positive definite kernel if for all $m > 0$ and $h_i \in H_i$, the following positivity condition holds: $$\sum_{-m \leq i\quad\, \atop j \leq m} \langle A(i,j) h_i, h_j \rangle \geq 0. $$

1. I wonder if the two definitions for positive-definite kernel agree with each other and how?
2. Is a positive-definite kernel related to a positive-definite bilinear form on a vector space?
3. What is the definition of kernel in its most general case, i.e. for the general Hilbert space case?

References are also appreciated. Thanks and regards!

Those definitions are very closely related, in that they can be put into the same general framework.

Let $X$ be a set, let $(H_x)_{x\in X}$ be a family of Hilbert spaces indexed by $X$, and for each $(x,y)\in X\times X$, let $K(x,y)$ be an element of $\mathcal{L}(H_x,H_y)$. Then $K$ is called a positive (semidefinite) kernel if for all finite sequences $x_1,\ldots x_n$ in $X$ and $h_1,\ldots,h_n$ with $h_i\in H_{x_i}$,

$$\sum_{i,j=1}^n\langle K(x_i,x_j)h_i,h_j\rangle\geq 0.$$ This is equivalent to requiring that the matrix $(K(x_j,x_i))_{ij}$ represents a positive operator on the Hilbert space $H_{x_1}\oplus\cdots\oplus H_{x_n}$. (There's a distracting transpose there, but I'm trying to make this consistent with the definition you gave.)

You get your second example by taking $X=\mathbb Z$. You get your first example by taking $X=[a,b]$ and $H_x=\mathbb R$ (as a real Hilbert space) for all $x$.

It is common (for example in the context mentioned below) that there is a single Hilbert space $H$ such that $H_x=H$ for all $x\in X$. In that case, $K:X\times X\to \mathcal{L}(H)$ is a positive kernel if for each finite sequence $x_1,\ldots,x_n$ in $X$, the matrix $(K(x_j,x_i))_{ij}$ represents a positive operator on the Hilbert space $H^{(n)}$.

So far I hope I have somewhat answered questions 1 and 3. Now I will discuss one context where such functions arise, leading to a partial answer to question 2.

One place where positive kernel functions arise is in the study of reproducing kernel Hilbert spaces. If $X$ is a set, $H$ is a Hilbert space, and $E$ is a Hilbert space whose elements are $H$-valued functions on $X$, then $E$ is called a (vector-valued) reproducing kernel Hilbert space if for each $x\in X$, evaluation at $x$ is a bounded linear operator from $E$ to $H$.

Suppose that $E$ satisfies this definition, and for each $x\in X$, let $\mathrm{ev}_x\in\mathcal{L}(E,H)$ be evaluation at $x$, $\mathrm{ev}_x(f)=f(x)$. Let $K:X\times X\to \mathcal{L}(H)$ be defined by $K(x,y)=\mathrm{ev}_x\mathrm{ev}_y^*$. Then $K$ is a positive kernel on $X$, called the reproducing kernel for $E$. The reason for the name "reproducing kernel" is that the evaluations of elements of $E$ can be "reproduced" from $K$ and the inner product on $E$. For each $x\in X$ and $h\in H$, the function $k_{x,h}:X\to H$ defined by $k_{x,h}(y)=K(y,x)h$ is in $E$. If $f$ is in $E$, then $\langle f(x),h\rangle=\langle f,k_{x,h}\rangle$. (In fact, note that $k_{x,h}=\mathrm{ev}_x^*h$.) This property uniquely characterizes $K$.

Conversely, if $X$ is a set, $H$ is a Hilbert space, and $K:X\times X\to \mathcal{L}(H)$ is a positive kernel function, then there is a unique reproducing kernel Hilbert space $E_K$ of $H$-valued functions on $X$ such that for all $f\in E$, $x\in X$, and $h\in H$, $k_{x,h}$ as defined above is in $E_K$, and $\langle f(x),h\rangle=\langle f,k_{x,h}\rangle$. In other words, $K$ is the (unique) reproducing kernel for a (unique) reproducing kernel Hilbert space. For the construction of $E_K$, a positive bilinear (or sesquilinear) form, i.e. an inner product, is defined on a free vector space whose formal generators "wind up" being the functions $k_{x,h}$ after completion. The positivity of $K$ is precisely what is needed to make the inner product work, so this might be a partial answer to your question 2. (I am leaving this part vague for now, but if you're interested I can elaborate or provide a reference.)

There are further generalizations that have appeared in operator theory and operator algebras literature, and there are certainly more types and applications of positive kernel functions than I am even aware of, let alone mentioned here. I may add more references at some point (I certainly will if you ask), but for now I will point out a couple that I have found particularly useful:

• Aronszajn's paper "Theory of reproducing kernels" contains the first systematic study of (scalar-valued) positive kernel functions and reproducing kernel Hilbert spaces.
• Agler and McCarthy's book Pick interpolation and Hilbert function spaces covers a number of topics related to positive kernel functions and reproducing kernel Hilbert spaces, particularly complex analytic interpolation problems and operator theory.

You might also be interested in a couple of past questions, here and here, which were about scalar-valued reproducing kernel Hilbert spaces (and hence, at least implicitly, involved positive scalar-valued kernels).