【Study Notes】Mathematics for Machine Learning: Linear Algebra (Week 3)

Introduction

This is a study note of the course Mathematics for Machine Learning: Linear Algebra on the coursera. In this part, we will learn what matrices is, and operations in matrices. We can make transformations by using mactrices, and find the inverse of a matrix. In addition, the determinants will be explaned in this section.

Matrix

How matrices transform space

Given a vector $r=\left[\begin{array}{c}x\\ y\end{array}\right]$ , and a $2\times 2$ matrix $A$ .
Start with $$Ar=r^{\prime }$$
We know $$r=\left[\begin{array}{c}x\\ y\end{array}\right]=x\widehat{e_1}+y\widehat{e_2}$$
where $\widehat{e_1}$ and $\widehat{e_2}$ are basic vectors, that is $\widehat{e_1}=\left[\begin{array}{c}1\\ 0\end{array}\right]$ and $\widehat{e_2}=\left[\begin{array}{c}0\\ 1\end{array}\right]$.
Then, we get$$\begin{array}{rl}Ar& =A(x\widehat{e_1}+y\widehat{e_2})\\ & =xA\widehat{e_1}+yA\widehat{e_2}\end{array}$$
So we have made a transformation by using matrix $A$. We change the basic vector $\widehat{e_1}$ to $A\widehat{e_1}$, and $\widehat{e_2}$ to $A\widehat{e_2}$.

If we assume matrix $A=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]$.
We get $A\widehat{e_1}=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]\left[\begin{array}{c}1\\ 0\end{array}\right]=\left[\begin{array}{c}a\\ c\end{array}\right]$, and $A\widehat{e_2}=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]\left[\begin{array}{c}0\\ 1\end{array}\right]=\left[\begin{array}{c}c\\ d\end{array}\right]$.
Finally, we change the basis to the columns of matrix $A$.
Then, we can say the basis after changing are the columns of matrix $A$.

Types of matrix transformations

Stretching

The matrix associated with a stretch by a factor $k$ along the x-axis is given by:
$$\left[\begin{array}{cc}k& 0\\ 0& 1\end{array}\right]$$
Similarly, a stretch by a factor $k$ along is given by:
$$\left[\begin{array}{cc}1& 0\\ 0& k\end{array}\right]$$

Squeezing

If the two stretches above are combined with reciprocal values, then the transformation matrix represents a squeeze mapping:
$$\left[\begin{array}{cc}k& 0\\ 0& 1/k\end{array}\right]$$
A square with sides parallel to axes is transformed to a rectangle that has the same area as the square. The reciprocal stretch and compression leave the area invariant.

Rotation

For rotation by an angle θ clockwise about the origin the functional form is $x^{\prime }=x\cos \theta +y\sin \theta$ and $y^{\prime }=-x\sin \theta +y\cos \theta$. Written in matrix form, this becomes:
$$\left[\begin{array}{cc}\cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{array}\right]$$
Similarly, for a rotation counterclockwise about the origin, the functional form is $x^{\prime }=x\cos \theta -y\sin \theta$ and $y^{\prime }=x\sin \theta +y\cos \theta$ the matrix form is:
$$\left[\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right]$$

Shearing

$$\left[\begin{array}{cc}1& k\\ 0& 1\end{array}\right]$$
or
$$\left[\begin{array}{cc}1& 0\\ k& 1\end{array}\right]$$