Introduction to Tikhonov Regularization Method

SCIENCE-BLOG
Regularization Inverse Problem

Notebook

First Steps with Regularization

Very short comment on Tikhonov regularization

Tikhonov regularization method

This method is widely used to resolve discrete ill-posed inverse linear problems (also non-linear). The problem it is usually establish as, \[y=F(x)\]

where F(x) is a linear operator, therefore we may write, \[y=Kx\]

The solution of the inverse discrete problem its usually described with the singular value decomposition (SVD). Then it is defined the pseudoinverse of K as, \[K^{\dagger}=(K^{T}K)^{-1}K^{T}=V\Sigma^{\dagger}U^{T}\]

The Tikhonov regularization propose another definition of the pseudoinverse which includes a regularization matrix \(L\) and a regularization parameter \(\alpha\), \[K^{\dagger}=(K^{T}K+\alpha L^{T}L)^{-1}K^{T}=\bar{V}\Sigma^{\dagger}_{\alpha}\bar{U}^{T}\]

In order to write the last equation the generalized singular value decomposition (GSVD) is used.

Measurements with noise1

The inversion method has an additional difficulty when the measurements incorporate errors, usually modelled as Gaussian Errors. In that case the vector state solution, written as \(x^{\epsilon}\) is related with the noisy measurements \(y^{\epsilon}\). The problem is formulated now as: \[y^{\epsilon}=Kx^{\epsilon}\]

and the additional problem is not just how to calculate the singular values, but how different are the estimation \(x^{\epsilon}\) and the true value \(x\).

However also there is a methodological problem on the definition of the solution \(x^{\epsilon}\) given that is not true that always \(y^{\delta}\in Im(K)\). Therefore, a new concept of solution is needed and it is usually formulated as an optimization rule: \[x^{\epsilon}: ||y^{\epsilon}-Kx^{\epsilon}||\le||y^{\epsilon}-Kx||\,\,\, \forall x \in \mathcal{X}\]

where \(\mathcal{X}\) is the domain of the Forward Model.

  1. Doicu, Adrian, Thomas Trautmann, and Franz Schreier. Numerical regularization for atmospheric inverse problems. Springer, 2010.