COMS 4771 HW0
Due: Sun Jan 27, 2019 at 11:59pm
This is a calibration assignment (HW0). The goal of this assignment is for you to recall basic
concepts, and get familiarized with the homework submission system (Gradescope). Everyone enrolled
or on the waitlist intending to enroll must submit this assignment by the due date. Anyone
who does not submit HW0 by the due date will get a score of zero. The score received on this assignment
will not count towards your final grade in this course, but will be used to make a decision
to who will be approved to enrolled. You must show your work to receive full credit. You should
cite all resources (including online material, books, articles, help taken from specific individuals,
etc.) you used to complete your work.
This homework assignment is to be done individually. All homeworks (including this one)
should be typesetted properly in pdf format. Handwritten solutions will not be accepted. You must
include your name and UNI in your homework submission.
0.1 [Notation]
Pr[·] denotes the probability (of an event).
E[·] denotes the expected value (of a random variable).
var[·] denotes the variance (of a random variable).
cov[·, ·] denotes the covariance (between a pair of random variables).
1[·] denotes the indicator function. That is, 1[A] := (
1 if A is true
0 otherwise .
⊥ denotes independence. That is, A⊥B means A and B are independent.
T denotes the transpose operator.
k · k denotes the Euclidean norm.
1.1 [Probability and Statistics]
Let X and Y be jointly distributed normal random variables, where
E[X] = 1, E[Y ] = 1,
var[X] = 1, var[Y ] = 9,
cov[X, Y ] = 2.
In other words, the joint distribution of the pair (X, Y ) ～ N(μ, Σ), where
What is the distribution of the random variable Z := X 2Y
1
1.2 Show that:
(i) For any (measurable) event A, we have Pr[A] = E[1[A]].
(ii) For any non-negative random variable X, and any c > 0
Pr[X ≥ c] ≤E[X]c.
(Hint: compare the output of the function 1[X > c] with the outcome of X.)
1.3 Let X ∈ {1, +1} denote the outcome of an toss of an unbiased coin. (That is, Pr[X =
+1] = Pr[X = 1] = 1/2.) Say the coin in tossed 1000 times independently, and the
correspoinding outcomes are denoted by X1, . . . , X1000.
Give a good estimate of the chance that the average of the 1000 tosses exceeds the value 10?
That is, give the best possible value of α, such that
Pr[(X1 + . . . + X1000) > 10] < α.
(Hint: use the result from q1.2.)
1.4 Suppose x is a random vector drawn from a d-dimensional multivariate Gassian distribution
with mean 0 and covariance Σ. Define y := Qx + v, for a known (invertible) d × d matrix Q,
and a d × 1 vector v. What is the distribution of y?
1.5 For any three random variables A, B and C prove or disprove the following statements:
(i) (A⊥B)|C =A⊥B.
(ii) A⊥B = (A⊥B)|C.
2.1 [Linear Algebra] Let v be a vector in R d
. Consider the set Sv := {x ∈ R d
| x · v = 0}.
What is the dimension of Sv? (Justify your answer).
2.2 Let Sv be as defined in q2.1, and w be another vector in R d
. What is the Euclidean distance
between w and the closest point to w in Sv? That is, find
min
x∈Sv
kw xk.
(for those who are curious, this minimum exists and you don’t need to prove its existance.)
2.3 Show that for any symmetric positive semi-definite d × d real matrix A, there exists real
vectors v1, . . . , vd such that
(Hint: consider the eigendecomposition of A)
2
3.1 [Calculus and optimization] For a given vector b ∈ R
d
, Consider the function Fb : (R
d×d × Rd) → R
defined as:
Fb : (A, v) 7→ v
TAv + b
(i) What is Fb/A
(ii) What is Fb/v
(iii) For a fixed invertible and symmetric matrix A, what value of v minimizes the function
Fb?
(iv) For a fixed invertible and symmetric matrix A, what value of v such that kvk = 1
minimizes the function Fb?
4.1 [Programming practice] Download the Matlab data file hw0data.mat (instructions on Piazza
on where to download the file). Write a script that does the following.
Special note for those who are not using Matlab: Python users can use scipy to read in
the mat file, R users can use R.matlab package to read in the mat file, Julia users can use
JuliaIO/MAT.jl. Octave users should be able to load the file directly.
(i) Load the data in hw0data.mat. It contains one matrix variable is called M.
(ii) Print the dimensions of M.
(iii) Print the 4th row and 5th column entry of M.
(iv) Print the mean value of the 5th column of M.
(v) Compute the histogram of the 4th row of M and show the figure.
(vi) Compute and print the top three eigenvalues of the matrix MTM.
4.2 We will try to understand the geometry of eigenvectors and eigenvalues of a matrix via experimentation.
Let L =
"
5/4 3/2
3/2 5 #
be a 2 × 2 matrix. To understand eigenvectors and
eigenvalues, we will study the action of L on random vectors and relate it to eigenvectors and
eigenvalues. Write a script that does the following.
(i) Create the 2 × 2 matrix L (as defined above).
(ii) Create 500 random, unit length, two-dimensional vectors. (Hint: to generate a random
d-dimensional unit length vector, draw d independent samples from the Gaussian distribution
N(0, 1) and assign each sample as one component of the vector. Now, normalize
the vector to have length one.) Let R be the set of these 500 random 2-dimensional unit
vectors.
(iii) For each vector r ∈ R, compute how the matrix L “distorts” r, that is, compute r := Lr.
(iv) Compute the eigenvalues of L. Let λmax and λmin denote the maximum and the minimum
eigenvalue respectively.
(v) For each distorted vector r, compute the length krk.
(vi) Create a histogram of values of k?rk (use 50 bins) and compare it to λmax and λmin.
(vii) What relationship can you infer between k?rk, λmax and λmin?
3
(viii) Now, compute the eigenvectors of L. Let vmax denote the eigenvector corresponding to
the maximum eigenvalue λmax.
(ix) Make a two-dimensional plot of all the distorted vectors r (in black color) and the eigenvector
Lvmax (in red color). (make sure that the x- and the y-axis are displayed at the
same scale).
(x) What can you infer about the vmax from studying this plot?
因为专业，所以值得信赖。如有需要，请加QQ：99515681 或邮箱：[email protected]
微信：codinghelp