digital-brain/content/zettels/norms.md

+++
title = "Systems and Signals Norms"
author = ["Thomas Dehaeze"]
draft = false
+++

Tags
:

\\[ \SI{1}{\meter\per\second} \\]

Resources:

-   ([Skogestad and Postlethwaite 2007](#orgf64cea0))
-   ([Toivonen 2002](#org3ddabae))
-   ([Zhang 2011](#orge43be7a))


## Definition {#definition}

<div class="definition">
  <div></div>

A norm of \\(e\\) (which may be a vector, matrix, signal of system) is a real number, denoted \\(\\|e\\|\\), that satisfies the following properties:

1.  Non-negative: \\(\\|e\\| \ge 0\\)
2.  Positive: \\(\\|e\\| = 0 \Longleftrightarrow e = 0\\)
3.  Homogeneous: \\(\\|\alpha \cdot e\\| = |\alpha| \cdot \\|e\\|\\) for all complex scalars \\(\alpha\\)
4.  Triangle inequality: \\(\\|e\_1 + e\_2\\| \le \\|e\_1\\| + \\|e\_2\\|\\)

</div>


## Vector Norms {#vector-norms}

-   **Vector 1-norm (Sum Norm)**:
    \\[ \\|a\\|\_1 \triangleq \sum\_i |a\_i| \\]
-   **Vector 2-norm (Euclidean Norm)**:
    \\[ \\|a\\|\_2 \triangleq \sqrt{\sum\_i |a\_i|^2} \\]
-   **Vector p-norm**:
    \\[ \\|a\\|\_p \triangleq \left( \sum\_i |a\_i|^p \right)^{1/p} \\]
-   **Vector \\(\infty\text{-norm}\\) (Max Norm)**: ()
    \\[ \\|a\\|\_\infty \triangleq \max\_i |a\_i| \\]


## Matrix Norms {#matrix-norms}

<div class="definition">
  <div></div>

A norm on a matrix \\(\\|A\\|\\) is a matrix norm if, in addition to the four norm properties, it also satisfies the multiplicative property:
\\[ \\|AB\\| \le \\|A\\| \cdot \\|B\\| \\]

</div>

-   **Sum matrix norm**:
    \\[ \\|A\\|\_\text{sum} \triangleq \sum\_{i,j} |a\_{ij}| \\]
-   **Frobenius matrix norm (Euclidean Norm)**:
    \\[ \\|A\\|\_F \triangleq \sqrt{\sum\_{i,j} |a\_{ij}|^2} = \sqrt{\text{tr}(A^H A)} \\]
-   **Max element norm**: (which is not a _matrix_ norm)
    \\[ \\|A\\|\_\text{max} \triangleq \max\_{i,j} |a\_{ij}| \\]


## Induced Matrix Norms {#induced-matrix-norms}

Induced matrix norms are important because of their close relationship to signal amplification in systems.

Consider the figure below where \\(w\\) is the input vector, \\(z\\) the output vector and where the "amplification" or "gain" of the matrix \\(A\\) is defined by the ration \\(\\|z\\|/\\|w\\|\\).

{{< figure src="/ox-hugo/induced_matrix_norm.png" >}}

The maximum gain for all possible input directions is given by the **induced norm**:
\\[ \\|A\\|\_{ip} \triangleq \max\_{w \neq 0} \frac{\\|Aw\\|\_p}{\\|w\\|\_p} \\]

Thus, the induced norm gives the largest possible "amplification" of the matrix.
The following equivalent definition is also used:
\\[ \\|A\\|\_{ip} = \max\_{\\|w\\|\_p \le 1} \\|Aw\\|\_p \\]


## Signal Norms {#signal-norms}

For signals, we may compute the norm in two steps:

1.  "Sum up" the channels at a given time using a vector norm.
    For a scalar, we simply take the absolute value.
2.  "Sum up" in time using a temporal norm.

We normally use the same p-norm both for the vector and the signal.

-   **1-norm in time (Integral Absolute Error)**:
    \\[ \\|e(t)\\|\_1 = \int\_{-\infty}^{\infty} \sum\_i |e\_i(\tau)| d\tau \\]
-   **2-norm in time (Quadratic Norm)**:
    \\[ \\|e(t)\\|\_2 = \sqrt{\int\_{-\infty}^{\infty} \sum\_i |e\_i(\tau)|^2 d\tau} \\]
-   **\\(\infty\text{-norm}\\) in time (Peak value in time)**:
    \\[ \\|e(t)\\|\_\infty = \max\_\tau \left( \max\_i |e\_i(\tau)| \right) \\]
-   **Power-Norm or RMS-Norm**:
    \\[ \\|e(t)\\|\_\text{pow} = \lim\_{T\to \infty} \sqrt{\frac{1}{2T} \int\_{-T}^T \sum\_i |e\_i(\tau)|^2 d\tau} \\]


## Signal Interpretation of Various System Norms {#signal-interpretation-of-various-system-norms}

Consider a system \\(G\\) with input \\(d\\) and output \\(e\\), such that:
\\[ e = G d \\]

For performance, we may want the output signal \\(e\\) to be "small" for any allowed input signals \\(d\\).
We therefore need to specify:

1.  What \\(d\\) are allowed. (Which set does \\(d\\) belong to?)
    Some possible inputs signal sets are:
    -   \\(d(t)\\) consists of impulses \\(\delta(t)\\).
    -   These generate step changes in the states.
    -   \\(d(t) = \sin(\omega t)\\) with fixed frequency
    -   \\(d(t)\\) is bounded in energy \\(\\|d(t)\\|\_2 \le 1\\)
    -   \\(d(t)\\) is bounded in power \\(\\|d(t)\\|\_\text{pow} \le 1\\)
    -   \\(d(t)\\) is bounded in magnitude \\(\\|d(t)\\|\_\infty \le 1\\)
2.  What we mean by "small". (Which norm should be use for \\(e\\)?)
    To measure the output signal, we may consider the following norms:
    -   2-norm (energy): \\(\\|e(t)\\|\_2\\)
    -   \\(\infty\text{-norm}\\) (peak magnitude): \\(\\|e(t)\\|\_\infty\\)
    -   Power: \\(\\|e(t)\\|\_\text{pow}\\)

We now consider which system norms result from the definition of input classes and output norms (Table [1](#table--tab:system-norms)).

<a id="table--tab:system-norms"></a>
<div class="table-caption">
  <span class="table-number"><a href="#table--tab:system-norms">Table 1</a></span>:
  System norms for sets of inputs signals and three different output norms
</div>

|                                           | \\(\delta(t)\\)                          | \\(\sin(\omega t)\\)                                   | \\(\vert\vert d \vert\vert\_2\\)         | \\(\vert\vert d \vert\vert\_\infty\\)        | \\(\vert\vert d \vert\vert\_\text{pow}\\) |
|-------------------------------------------|------------------------------------------|--------------------------------------------------------|------------------------------------------|----------------------------------------------|-------------------------------------------|
| \\(\vert\vert e \vert\vert\_2\\)          | \\(\vert\vert G(s) \vert\vert\_2\\)      | \\(\infty\\)                                           | \\(\vert\vert G(s) \vert\vert\_\infty\\) | \\(\infty\\)                                 | \\(\infty\\)                              |
| \\(\vert\vert e \vert\vert\_\infty\\)     | \\(\vert\vert g(t) \vert\vert\_\infty\\) | \\(\overline{\sigma}(G(j\omega))\\)                    | \\(\vert\vert G(s) \vert\vert\_2\\)      | \\(\vert\vert g(t) \vert\vert\_1\\)          | \\(\infty\\)                              |
| \\(\vert\vert e \vert\vert\_\text{pow}\\) | 0                                        | \\(\frac{1}{\sqrt{2}} \overline{\sigma}(G(j\omega))\\) | 0                                        | \\(\le \vert\vert G(s) \vert\vert\_\infty\\) | \\(\vert\vert G(s) \vert\vert\_\infty\\)  |


## System Norms {#system-norms}


### \\(\mathcal{H}\_\infty\\) Norm {#mathcal-h-infty--norm}

<div class="exampl">
  <div></div>

Consider a proper linear stable system \\(G(s)\\).
The \\(\mathcal{H}\_\infty\\) norm is the peak value of its maximum singular value:
\\[ \\|G(s)\\|\_\infty \triangleq \max\_{\omega} \overline{\sigma}(G(j\omega)) \\]

</div>

In terms of signals, the \\(\mathcal{H}\_\infty\\) norm can be interpreted as follows:

-   it is the worst case steady-state gain for sinusoidal inputs at any frequency
-   it is equal to the 2-norm in the time domain:
    \\[ \\|G(s)\\|\_\infty = \max\_{d(t)} \frac{\\|e(t)\\|\_2 \neq 0}{\\|d(t)\\|\_2} = \max\_{\\|d(t)\\|\_2 = 1} \\|e(t)\\|\_2 \\]


### \\(\mathcal{H}\_2\\) Norm {#mathcal-h-2--norm}

<div class="exampl">
  <div></div>

Consider a strictly proper system \\(G(s)\\).
The \\(\mathcal{H}\_2\\) norm is:

\begin{align\*}
\\|G(s)\\|\_2 &\triangleq \sqrt{\frac{1}{2\pi} \int\_{-\infty}^{\infty} \text{tr}\left(G(j\omega)^HG(j\omega)\right) d\omega} \\\\\\
           &= \sqrt{\frac{1}{2\pi} \int\_{-\infty}^{\infty} \sum\_i {\sigma\_i}^2(G(j\omega)) d\omega}
\end{align\*}

</div>

In terms of signals, the \\(\mathcal{H}\_\infty\\) norm can be interpreted as follows:

-   it is a measure of the expected RMS value of the output to white noise excitation

The \\(\mathcal{H}\_2\\) is very useful when combined to [Dynamic Error Budgeting]({{< relref "dynamic_error_budgeting" >}}).

As explained in ([Monkhorst 2004](#org45aca82)), the \\(\mathcal{H}\_2\\) norm has a stochastic interpretation:

> The squared \\(\mathcal{H}\_2\\) norm can be interpreted as the output variance of a system with zero mean white noise input.


## Bibliography {#bibliography}

<a id="org45aca82"></a>Monkhorst, Wouter. 2004. “Dynamic Error Budgeting, a Design Approach.” Delft University.

<a id="orgf64cea0"></a>Skogestad, Sigurd, and Ian Postlethwaite. 2007. _Multivariable Feedback Control: Analysis and Design_. John Wiley.

<a id="org3ddabae"></a>Toivonen, Hannu T. 2002. “Robust Control Methods.” Abo Akademi University.

<a id="orge43be7a"></a>Zhang, Weidong. 2011. _Quantitative Process Control Theory_. CRC Press.