Normalized Least Mean Square

$\vec{a}:\text{the unknown vector of the system parameters}$
$\vec{x}:\text{the vector of the input signal }$
$y:\text{the output signal }$

$y=\vec{a}\cdot\vec{x}$

$\vec{a}^{\prime}:\text{the vector of the prior estimated parameters}$

$\text{the estimated output}: y^{\prime}=\vec{a}^{\prime}\cdot\vec{x}$

$\text{error}:e=y-y^{\prime}$

$\vec{a}^*:\text{the vector of the posterior estimated parameters}$

$\text{assuming } y=\vec{a}^*\cdot\vec{x} \text{ and } \vec{a}^*=\vec{a}^{\prime}+\mu\vec{x}$

$e=(\vec{a}^{\prime}+\mu\vec{x})\cdot\vec{x} - \vec{a}^{\prime}\cdot\vec{x}$

$e=\mu\vec{x}\cdot\vec{x}=\mu{\lVert\vec{x}\rVert}^2$

$\mu=\dfrac{e}{{\lVert\vec{x}\rVert}^2}$

$\vec{a}^*=\vec{a}^{\prime}+\dfrac{e}{{\lVert\vec{x}\rVert}^2}\vec{x}$

Technology and Business Skills

$Result = {Technology} ^ {Business Skills}$

Technology	Business Skills	Result
0	*	0
1	0	1
1	1	1
1	2	1
2	0	1
2	1	2
2	2	4
2	3	8

Estimation - Kalman Filter II

From the point of view of the measurement, we can make a prediction of the measurement ${\textbf{z}_1}'$ from ${\textbf{x}_1}'$, and we can get $({\textbf{z}_1}', \textbf{H}_1{\textbf{P}_1}'\textbf{H}_1^T)$.

The measurement is $(\textbf{z}_1, \textbf{R}_1)$, the estimate is

$\textbf{z}_1^* = (\textbf{I}-\textbf{G}_1){\textbf{z}_1}'+\textbf{G}_1\textbf{z}_1$

a good estimate comes with

$\textbf{G}_1=\textbf{H}_1{\textbf{P}_1}'\textbf{H}_1^T(\textbf{H}_1{\textbf{P}_1}'\textbf{H}_1^T+\textbf{R}_1)^{-1} = \textbf{H}_1\textbf{K}_1$

$\textbf{z}_1^* = (\textbf{I}-\textbf{H}_1\textbf{K}_1)\textbf{H}_1{\textbf{x}_1}'+\textbf{H}_1\textbf{K}_1\textbf{z}_1$

$ = \textbf{H}_1(\textbf{I}-\textbf{K}_1\textbf{H}_1){\textbf{x}_1}'+\textbf{H}_1\textbf{K}_1\textbf{z}_1$

$\textbf{H}_1\textbf{x}_1^* = \textbf{H}_1(\textbf{I}-\textbf{K}_1\textbf{H}_1){\textbf{x}_1}'+\textbf{H}_1\textbf{K}_1\textbf{z}_1$

so $\textbf{K}_1$ can give a good estimate for $\textbf{z}_1^*$, it seems also imply that

 $(\textbf{I}-\textbf{K}_1\textbf{H}_1){\textbf{x}_1}'+\textbf{K}_1\textbf{z}_1$ can give a good estimate for $\textbf{x}_1^*$.