Suggested languages for you:

Americas

Europe

Q7.5-12E

Expert-verified
Found in: Page 395

### Linear Algebra and its Applications

Book edition 5th
Author(s) David C. Lay, Steven R. Lay and Judi J. McDonald
Pages 483 pages
ISBN 978-03219822384

# Question: Let $${\bf{X}}$$ denote a vector that varies over the columns of a $$p \times N$$ matrix of observations, and let $$P$$ be a $$p \times p$$ orthogonal matrix. Show that the change of variable $${\bf{X}} = P{\bf{Y}}$$ does not change the total variance of the data. (Hint: By Exercise 11, it suffices to show that $$tr\left( {{P^T}SP} \right) = tr\left( S \right)$$. Use a property of the trace mentioned in Exercise 25 in Section 5.4.)

It is verified that the total variance would not change when variables change as $${\bf{X}} = P{\bf{Y}}$$.

See the step by step solution

## Step 1: Mean Deviation form and Covariance Matrix.

The Mean Deviation form of any $$p \times N$$ is given by:

$$B = \left( {\begin{array}{*{20}{c}}{{{{\bf{\hat X}}}_1}}&{{{{\bf{\hat X}}}_2}}&{........}&{{{{\bf{\hat X}}}_N}}\end{array}} \right)$$

Whose $$p \times p$$ covariance matrix is:

$$S = \frac{1}{{N - 1}}B{B^T}$$

## Step 2: The Variance

From exercise 11, we have:

$${S_Y} = {P^T}SP$$

When variable changes as: $${\bf{X}} = P{\bf{Y}}$$

The traces of the covariance matrices $${S_Y}{\rm{ and }}S$$ will be the same.

The total variance of the data is given by $${\bf{Y}}$$ is $${\rm{tr}}\left( {{P^T}SP} \right)$$.

For two similar matrices $$A,B$$ are such that, $${\bf{B}} = P{\bf{A}}{P^{ - 1}}$$ which implies $${\rm{tr}}\left( {\bf{B}} \right) = {\rm{tr}}\left( {P{\bf{A}}{P^{ - 1}}} \right)$$.

In the obtained equation, if $$P$$ is an orthogonal matrix, then $${P^T} = {P^{ - 1}}$$.

Apply trace on both sides of $${P^T} = {P^{ - 1}}$$ and simplify.

$$\begin{array}{c}{\rm{tr}}\left( {{P^T}SP} \right) = {\rm{tr}}\left( {{P^{ - 1}}SP} \right)\\ = {\rm{tr}}\left( {{P^{ - 1}}PS} \right)\\ = {\rm{tr}}\left( {\left( {{P^{ - 1}}P} \right)S} \right)\\ = {\rm{tr}}\left( {IS} \right)\\ = {\rm{tr}}\left( S \right)\end{array}$$

Thus, the total variance would not change when variables change as: $${\bf{X}} = P{\bf{Y}}$$.

Hence, this is the required proof.