Angle bracket and sharp bracket for discontinuous processes
Solution 1:
Let $(X_t,\mathcal{F}_t)_{t \geq 0}$ be an (càdlàg) $L^2$-martingale, i.e. a martingale which satisfies
$$\sup_{t < \infty} \mathbb{E}(X_t^2)<\infty.$$
Then it follows from the Doob-Meyer decomposition that there exists a unique increasing previsible process $(A_t)_{t \geq 0}$ such that $A_0=0$ and
$$(X_t^2-A_t,\mathcal{F}_t)_{t \geq 0} \,\, \text{is a martingale}. \tag{1}$$
The process $A_t:= \langle X \rangle_t$ is called angle bracket or previsible quadratic variation. If $(X_t)_t$ and $(Y_t)_t$ are two martingales, then the covariation is defined via polarization, i.e.
$$\langle X,Y \rangle_t := \frac{1}{4} \big( \langle X+Y \rangle_t - \langle X-Y \rangle_t \big). \tag{2}$$
This definition implies that $(X_t Y_t - \langle X,Y \rangle_t,\mathcal{F}_t)_{t \geq 0}$ is a martingale. In particular, the notion of predictable quadratic variation is restricted to martingales whereas the sharp bracket is defined for semimartingales: For a semimartingale $(X_t)_{t \geq 0}$, we set
$$[X]_t := X_t^2-X_0^2 - 2 \int_0^t X_{s-} \, dX_s. \tag{3}$$
The covariation $[X,Y]_t$ is again defined via polarization. Another characterization of the sharp bracket is the following:
$$[X,Y]_t = \text{ucp}-\lim_{k \to \infty} \sum_{j=0}^{n-1} (X_{t_{j+1}^k \wedge t}-X_{t_j^k \wedge t}) \cdot (Y_{t_{j+1}^k \wedge t}-Y_{t_j^k \wedge t})$$
where the partitions $\pi_k = \{0=t_0^k < \ldots < t_n^k<\infty\}$ satisfy $|\pi_k| \to 0$ (Here ucp denotes the uniform [with respect to $t$] limit in probability.) See e.g. Protter [2] for a proof.
If $(X_t)_{t \geq 0}$ is a martingale, then
$$X_t^2 - [X]_t = 2 \int_0^t X_{s-} \, dX_s$$
is a martingale, i.e. $(1)$ is satisfied. It is important to note that this does not imply $[X]_t = \langle X \rangle_t$; in fact, the process $[X]_t$ is in general not previsible. One (important) exception are martingales with continuous sample paths. In fact, for continuous martingales $(X_t)_t$ it holds that $[X]=\langle X \rangle$. On the other hand, any $L^2$-martingale $(X_t)_{t \geq 0}$ admits a decomposition of the form
$$X_t = X_t^c+X_t^d$$
where $(X_t^c)_{t \geq 0}$ is a continuous martingale and $(X_t^d)_{t \geq 0}$ a pure-jump martingale. One can show that
$$[X]_t = [X^c]_t + [X^d]_t = \langle X^c \rangle_t+ \sum_{s \leq t} (\Delta X_s)^2. \tag{4}$$
There are several books which introduce both notions of quadratic variation, but there are only few containing a proof of the (very important) equality $(4)$. One of them is the monograph 1 by Jacod and Shiryaev.
Let me finish this answer with some basic examples:
Example 1: Brownian motion
If $(X_t)_{t \geq 0}$ is a Brownian motion, then $(X_t)_{t \geq 0}$ is a martingale and it is not difficult to see that $\langle X \rangle_t = t$. Since Brownian motion is continuous, we get $[X]_t = t$.
Example 2: Subordinate Brownian motion
Let $(B_t)_{t \geq 0}$ be a Brownian motion and $f:[0,\infty) \to [0,\infty)$ an increasing càdlàg function. Then $X_t := B_{f(t)}$ is called a (particular case of a) subordinate Brownian motion. Using $(1)$ and the independence and stationarity of the increments, it is not difficult to see that $\langle X \rangle_t = f(t)$. A direct (lenghty) calculation reveals that $$[X]_t = f(t) - \sum_{s \leq t} \Delta f(s) + \sum_{s \leq t} |\Delta X_s|^2.$$ If we choose $f(t) = t$ we recover Example 1.
Example 3: Poisson process
Let $(X_t)_{t \geq 0}$ be a compensated Poisson process with intensity $\lambda>0$. Using the independence and stationarity of the increments, we see that $\langle X \rangle_t = \lambda t$. In particular, $\langle X \rangle_t$ is deterministic. On the other hand, it follows from $(4)$ that $$[X]_t = \sum_{s \leq t} |\Delta X_s|^2.$$ This is one of the easiest examples showing that both notions of quadratic variation are (in general) totally different.
Example 4: Square integrable martingales
Let $(M_t)_{t \geq 0}$ be an $L^2$-martingale and $f$ a "nice" function such that the stochastic integral $X_t := \int_0^t f(s) \, dM_s$ is well-defined. Then $$\langle X \rangle_t = \int_0^t f(s)^2 \, d\langle M \rangle_s.$$ A similar result can be obtained for stochastic integrals with respect to Poisson random measures.
Literature:
- (1) Limit Theorems for Stochastic Processes - J. Jacod, A. Shiryaev [Angle Bracket, Sharp Bracket]. This book contains all mentioned results (and proofs thereof).
- (2) Stochastic Integration and Differential Equations - P. Protter [Sharp Bracket]
- (3) Continuous Martingales and Brownian Motion - D. Revuz, M. Yor [Angle Bracket]
- (4) Stochastic Differential Equations and Diffusion Processes - N. Ikeda, S.Watanabe [Angle Bracket]
- (5) Brownian Motion and Stochastic Calculus - I. Karatzas, S. Shreve [Angle Bracket]