Regularization

Notebook

### Introduction to Tikhonov Regularization Method

First Steps with Regularization

SCIENCE-BLOG

Very short comment on Tikhonov regularization

## Tikhonov regularization method

This method is widely used to resolve discrete ill-posed inverse linear problems (also non-linear). The problem it is usually establish as, $y=F(x)$

where F(x) is a linear operator, therefore we may write, $y=Kx$

The solution of the inverse discrete problem its usually described with the singular value decomposition (SVD). Then it is defined the pseudoinverse of K as, $K^{\dagger}=(K^{T}K)^{-1}K^{T}=V\Sigma^{\dagger}U^{T}$

The Tikhonov regularization propose another definition of the pseudoinverse which includes a regularization matrix $$L$$ and a regularization parameter $$\alpha$$, $K^{\dagger}=(K^{T}K+\alpha L^{T}L)^{-1}K^{T}=\bar{V}\Sigma^{\dagger}_{\alpha}\bar{U}^{T}$

In order to write the last equation the generalized singular value decomposition (GSVD) is used.

### Measurements with noise1

The inversion method has an additional difficulty when the measurements incorporate errors, usually modelled as Gaussian Errors. In that case the vector state solution, written as $$x^{\epsilon}$$ is related with the noisy measurements $$y^{\epsilon}$$. The problem is formulated now as: $y^{\epsilon}=Kx^{\epsilon}$

and the additional problem is not just how to calculate the singular values, but how different are the estimation $$x^{\epsilon}$$ and the true value $$x$$.

However also there is a methodological problem on the definition of the solution $$x^{\epsilon}$$ given that is not true that always $$y^{\delta}\in Im(K)$$. Therefore, a new concept of solution is needed and it is usually formulated as an optimization rule: $x^{\epsilon}: ||y^{\epsilon}-Kx^{\epsilon}||\le||y^{\epsilon}-Kx||\,\,\, \forall x \in \mathcal{X}$

where $$\mathcal{X}$$ is the domain of the Forward Model.

1. Doicu, Adrian, Thomas Trautmann, and Franz Schreier. Numerical regularization for atmospheric inverse problems. Springer, 2010.