digital-brain/content/zettels/singular_value_decomposition.md

+++
title = "Singular Value Decomposition"
author = ["Thomas Dehaeze"]
draft = false
+++

Tags
:


## SVD of a MIMO system {#svd-of-a-mimo-system}

This is taken from ([Skogestad and Postlethwaite 2007](#org4953f60)).

We are interested by the physical interpretation of the SVD when applied to the frequency response of a MIMO system \\(G(s)\\) with \\(m\\) inputs and \\(l\\) outputs.

\begin{equation}
G = U \Sigma V^H
\end{equation}

-   \\(\Sigma\\): is an \\(l \times m\\) matrix with \\(k = \min\\{l, m\\}\\) non-negative **singular values** \\(\sigma\_i\\), arranged in descending order along its main diagonal, the other entries are zero.
-   \\(U\\): is an \\(l \times l\\) unitary matrix. The columns of \\(U\\), denoted \\(u\_i\\), represent the **output directions** of the plant. They are orthonormal.
-   \\(V\\): is an \\(m \times m\\) unitary matrix. The columns of \\(V\\), denoted \\(v\_i\\), represent the **input directions** of the plant. They are orthonormal.

The input and output directions are related through the singular values:

\begin{equation}
  G v\_i = \sigma\_i u\_i
\end{equation}

So, if we consider an input in the direction \\(v\_i\\), then the output is in the direction \\(u\_i\\).
Furthermore, since \\(\\|v\_i\\|\_2=1\\) and \\(\\|u\_i\\|\_2=1\\), we see that **the singular value \\(\sigma\_i\\) directly gives the gain of the matrix \\(G\\) in this direction**.

The **largest gain** for any input is equal to the **maximum singular value**:
\\[\overline{\sigma}(G) \equiv \sigma\_1(G) = \max\_{d\neq 0}\frac{\\|Gd\\|\_2}{\\|d\\|\_2} = \frac{\\|Gv\_1\\|\_2}{\\|v\_1\\|\_2} \\]
The **smallest gain** for any input direction is equal to the **minimum singular value**:
\\[ \underline{\sigma}(G) \equiv \sigma\_k(G) = \min\_{d\neq 0}\frac{\\|Gd\\|\_2}{\\|d\\|\_2} = \frac{\\|Gv\_k\\|\_2}{\\|v\_k\\|\_2} \\]

We define \\(u\_1 = \overline{u}\\), \\(v\_1 = \overline{v}\\), \\(u\_k=\underline{u}\\) and \\(v\_k = \underline{v}\\).
Then is follows that:
\\[ G\overline{v} = \overline{\sigma} \cdot \overline{u} ; \quad G\underline{v} = \underline{\sigma} \cdot \underline{u} \\]


## SVD to pseudo inverse rectangular matrices {#svd-to-pseudo-inverse-rectangular-matrices}

This is taken from ([Preumont 2018](#org6558f35)).

The **Singular Value Decomposition** (SVD) is a generalization of the eigenvalue decomposition of a rectangular matrix:
\\[ J = U \Sigma V^T = \sum\_{i=1}^r \sigma\_i u\_i v\_i^T \\]
With:

-   \\(U\\) and \\(V\\) orthogonal matrices. The columns \\(u\_i\\) and \\(v\_i\\) of \\(U\\) and \\(V\\) are the eigenvectors of the square matrices \\(JJ^T\\) and \\(J^TJ\\) respectively
-   \\(\Sigma\\) a rectangular diagonal matrix of dimension \\(m \times n\\) containing the square root of the common non-zero eigenvalues of \\(JJ^T\\) and \\(J^TJ\\)
-   \\(r\\) is the number of non-zero singular values of \\(J\\)

The pseudo-inverse of \\(J\\) is:
\\[ J^+ = V\Sigma^+U^T = \sum\_{i=1}^r \frac{1}{\sigma\_i} v\_i u\_i^T \\]

The conditioning of the Jacobian is measured by the **condition number**:
\\[ c(J) = \frac{\sigma\_{max}}{\sigma\_{min}} \\]

When \\(c(J)\\) becomes large, the most straightforward way to handle the ill-conditioning is to truncate the smallest singular value out of the sum.
This will have usually little impact of the fitting error while reducing considerably the actuator inputs \\(v\\).


## Bibliography {#bibliography}

<a id="org6558f35"></a>Preumont, Andre. 2018. _Vibration Control of Active Structures - Fourth Edition_. Solid Mechanics and Its Applications. Springer International Publishing. <https://doi.org/10.1007/978-3-319-72296-2>.

<a id="org4953f60"></a>Skogestad, Sigurd, and Ian Postlethwaite. 2007. _Multivariable Feedback Control: Analysis and Design_. John Wiley.