Variance-Covariance Matrix

This lesson explains how to use matrix methods to generate a variance-covariance matrix from a matrix of raw data.

Variance

Variance is a measure of the variability or spread in a set of data. Mathematically, it is the average squared deviation from the mean score. We use the following formula to compute variance.

Var(X) = Σ ( X_i - X )² / N = Σ x_i² / N

where

N is the number of scores in a set of scores
X is the mean of the N scores.
X_i is the ith raw score in the set of scores
x_i is the ith deviation score in the set of scores
Var(X) is the variance of all the scores in the set

Covariance

Covariance is a measure of the extent to which corresponding elements from two sets of ordered data move in the same direction. We use the following formula to compute covariance.

Cov(X, Y) = Σ ( X_i - X ) ( Y_i - Y ) / N = Σ x_iy_i / N

where

N is the number of scores in each set of data
X is the mean of the N scores in the first data set
X_i is the ithe raw score in the first set of scores
x_i is the ith deviation score in the first set of scores
Y is the mean of the N scores in the second data set
Y_i is the ithe raw score in the second set of scores
y_i is the ith deviation score in the second set of scores
Cov(X, Y) is the covariance of corresponding scores in the two sets of data

Variance-Covariance Matrix

Variance and covariance are often displayed together in a variance-covariance matrix, (aka, a covariance matrix). The variances appear along the diagonal and covariances appear in the off-diagonal elements, as shown below.

V    =

Σ x12 / N     Σ x1 x2 / N     . . .     Σ x1 xc / N   Σ x2 x1 / N     Σ x22 / N     . . .     Σ x2 xc / N . . .     . . .     . . .     . . . Σ xc x1 / N     Σ xc x2 / N     . . .     Σ xc2 / N

where

V is a c x c variance-covariance matrix
N is the number of scores in each of the c data sets
x_i is a deviation score from the ith data set
Σ x_i² / N is the variance of elements from the ith data set
Σ x_i x_j / N is the covariance for elements from the ith and jth data sets