Minimization of Variational - Total Variation (TV) Deblurring

Under the Linear Blurring Model - $ f = H \ast u $.
I'm trying to calculate the Euler Lagrange of with respect to $ u $ of the functional:

$$ E \left( u \right) = {\left\| f - H \ast u \right\|}^{2}_{2} $$

I know the answer should be $ H \left( -x \right) \ast \left( f - H \ast u \right) $.
Since is must be equivalent to the discrete solution of Linear Operator.
Namely for $ J \left( u \right) = {\left\| f - A u \right\|}^{2}_{2} $ the functional derivative is given by $ {J}' \left( u \right) = {A}^{T} \left( f - A u \right) $.

This is result of the Adjoint of the Convolution is the mirroring the function - $ \bar{f} \left( x \right) = f \left( - x \right) $.

Still I don't understand how to apply it and how to calculate the derivative with respect to $ u $, a function, under the Convolution Operator.

Thank You.

Solution 1:

Define

$$G(x) := (H \ast u)(x)=\int_{-\infty}^{\infty}H(x-s)u(s)\,ds,\\\hat{H}(x) := H(-x).$$

Then

$$E(u) = {\left\| f - H \ast u \right\|}^{2}_{2}=\int_{-\infty}^{\infty}\left|f(x)-G(x)\right|^2\,dx\\ = \int_{-\infty}^{\infty}\left|f(x)-\int_{-\infty}^{\infty}H(x-s)u(s)\,ds\right|^2\,dx.$$

The variational (Gateaux) derivative for some admissible function $v$ is

$$D_vE(u)= \left.\frac{d}{d \epsilon}E(u + \epsilon v)\right|_{\epsilon = 0} \\ = \int_{-\infty}^{\infty}2\left[f(x)-G(x)\right]\left[\int_{-\infty}^{\infty}H(x-s)v(s)\,ds\right]\,dx \\ = \int_{-\infty}^{\infty}2\left[\int_{-\infty}^{\infty}H(x-s)\left[f(x)-G(x)\right]\,dx\right]v(s)\,ds \\ = \int_{-\infty}^{\infty}2\left[\int_{-\infty}^{\infty}\hat{H}(s-x)\left[f(x)-G(x)\right]\,dx\right]v(s)\,ds \\ = \int_{-\infty}^{\infty}2\hat{H} \ast (f - H \ast u)(s)v(s)\,ds.$$

A necessary condition for an extremum is that the variational derivative vanish for all admissible $v$:

$$\int_{-\infty}^{\infty}2\hat{H} \ast (f - H \ast u)(s)v(s)\,ds = 0 \\ \implies\hat{H} \ast (f - H \ast u) \equiv 0$$

As an aside, the variational derivative is analagous to the directional derivative of a function defined on a finite-dimensional linear space. A function, $f: \mathbb{R^n}\to \mathbb{R^m}$, is differentiable at a point $a$ if there exists a linear operator $Df[a]: \mathbb{R^n}\to \mathbb{R^m}$ such that for $h \in \mathbb{R^n}$

$$\lim_{||h|| \to 0}\frac{||f(a+h) - f(a) - Df[a]\cdot h||}{||h||} =0.$$

The directional derivative, in the direction $v \in \mathbb{R^n}$, is defined as the element $D_vf(a) \in \mathbb{R^m}$ such that for each component

$$\lim_{t \to 0}t^{-1}[f_i(a+tv) - f_i(a)] = D_vf_i(a).$$

The existence of the directional derivative is necessary but not sufficient for the existence of the derivative operator. However, when both exist

$$D_vf(a) = Df[a] \cdot v.$$

In the setting of this problem, the variational derivative of the functional $E$ is an element of the ambient space: $D_vE[u] \in L_2(\mathbb{R})$ with $u,v \in L_2(\mathbb{R})$. This represents a directional derivative in the "direction" $v$. The variation of $E$ is the linear operator $\delta E[u]: L_2(\mathbb{R}) \to L_2(\mathbb{R}) $ defined as

$$\delta E[u]: v \mapsto \int_{-\infty}^{\infty}2\hat{H} \ast (f - H \ast u)(s)v(s)\,ds = D_vE[u].$$