Review 6

Today we talked about linear regression. In linear regression, the machine learning system predicts a numerical value in R given numerical inputs(features) in real valued space R. In other words, we determine a function or mapping F. In classification, we find y=f(X) that maps X to one of the k classes [c1 c2 ... ck].

K-Nearest Neighbor. Let us consider R^2 as shown below. The points shown in black are training data set(in other words we know the class X0 fir the given points.) First consider a simple case of k=1 where we will determine class of the new circles. In this case, we determine class of the new point as the class of points in the circle, which in our case is 1. 

Determine computational complexity of K-Nearest Neighbor for:

n training points

m dimensions

Euclidean dissimilarity measure

How many mathematical operations will be needed. And we discussed how many mathematical operations will be needed to calculate the distance. 

Then, professor introduced the Iris data set. It has been downloaded for over 1 million times. He showed the KNN- Confusion Matrices. And it has many precision, recall and F1 scores. 

Gradient Descent

A convex function is a function which means that it has a single minimum point. And if a function has multiple local minima, it is not a convex function.

Slope 

positive and negative slope 

and we can use calculus to 

show the update formula the goal is to find out the minimum of the function.  This is achieved by choosing a random point x on the horizon.