The Matrix format of Least Square Method (LMS)

I. Cost function

For the cost function

$$J=\sum _{i=1}^n||y_i-x_i^T\theta |{|}^2\text{\hspace{1em}(1)}$$

where $\theta (m\times 1)$ is the unknow parameters, $x_i^T(1\times m)$ and $y_i(1\times 1)$ are the collected data.

II. The optimal solution

The optimal estimation of $\theta$ is

$$\hat{\theta }=(X^{T}X{)}^{-1}{X}^{T}Y\text{\hspace{1em}(2)}$$

where $Y(n\times 1)$ and $X(n\times m)$ as follows

Y = [ y 1 ⋮ y n ] ( n × 1 ) X = [ x 1 T ⋮ x n T ] ( n × m ) (3) Y = \left[\begin{array}{c} y_1 \\ \vdots \\ y_n \end{array}\right]_{(n\times 1)} \qquad X = \left[\begin{array}{c} x_1^T \\ \vdots \\ x_n^T \end{array}\right]_{(n\times m)} \tag{3} Y= ​y1​⋮yn​​ ​(n×1)​X= ​x1T​⋮xnT​​ ​(n×m)​(3)

III. Proof

Omiting.

— over —