23.5. Exercises#

1. A random vector \(\mathbf{Y} = [Y_1 ~~ Y_2 ~~ \cdots ~~ Y_n]^T\) has mean vector \(\boldsymbol{\mu}\) and covariance matrix \(\sigma^2 \mathbf{I}_n\) where \(\sigma > 0\) is a number and \(\mathbf{I}_n\) is the \(n \times n\) identity matrix.

(a) Pick one option and explain: \(Y_1\) and \(Y_2\) are

\(~~~~~\) (i) independent. \(~~~~~~~~\) (ii) uncorrelated but might not be independent. \(~~~~~~~~\) (iii) not uncorrelated.

(b) Pick one option and explain: \(Var(Y_1)\) and \(Var(Y_2)\) are

\(~~~~~\) (i) equal. \(~~~~~~~~\) (ii) possibly equal, but might not be. \(~~~~~~~~\) (iii) not equal.

(c) For \(m \le n\) let \(\mathbf{A}\) be an \(m \times n\) matrix of real numbers, and let \(\mathbf{b}\) be an \(m \times 1\) vector of real numbers. Let \(\mathbf{V} = \mathbf{AY} + \mathbf{b}\). Find the mean vector \(\boldsymbol{\mu}_\mathbf{V}\) and covariance matrix \(\boldsymbol{\Sigma}_\mathbf{V}\) of \(\mathbf{V}\).

(d) Let \(\mathbf{c}\) be an \(m \times 1\) vector of real numbers and let \(W = \mathbf{c}^T\mathbf{V}\) for \(\mathbf{V}\) defined in Part (c). In terms of \(\mathbf{c}\), \(\boldsymbol{\mu}_\mathbf{V}\) and \(\boldsymbol{\Sigma}_\mathbf{V}\), find \(E(W)\) and \(Var(W)\).

2. Let \([U ~ V ~ W]^T\) be multivariate normal with mean vector \([0 ~ 0 ~ 0]^T\) and covariance matrix \(\begin{bmatrix} 1 & \rho_1 & \rho_2 \\ \rho_1 & 1 & \rho_3 \\ \rho_2 & \rho_3 & 1 \end{bmatrix}\)

(a) What is the distribution of \(U\)?

(b) What is the distribution of \(U+2V\)?

(c) What is the joint distribution of \(U\) and \(U+2V\)?

(d) Under what condition on the parameters is \(U\) independent of \(U+2V\)?

3. Let \([X_1 ~~ X_2 ~~ X_3]^T\) be multivariate normal with mean vector \(\boldsymbol{\mu}\) and covariance matrix \(\boldsymbol{\Sigma}\) given by

\[\begin{split} \boldsymbol{\mu} ~ = ~ \begin{bmatrix} \mu \\ \mu \\ \mu \end{bmatrix} ~~~~~~~~~~~ \boldsymbol{\Sigma} ~ = ~ \begin{bmatrix} \sigma_1^2 & \sigma_{12} & \sigma_{13} \\ \sigma_{21} & \sigma_2^2 & \sigma_{23} \\ \sigma_{31} & \sigma_{32} & \sigma_3^2 \end{bmatrix} \end{split}\]

Find \(P\big{(} (X_1 + X_2)/2 < X_3 + 1 \big{)}\).

4. Let \(X\) be standard normal. Construct a random variable \(Y\) as follows:

  • Toss a fair coin.

  • If the coin lands heads, let \(Y = X\).

  • If the coin lands tails, let \(Y = -X\).

(a) Find the cdf of \(Y\) and hence identify the distribution of \(Y\).

(b) Find \(E(XY)\) by conditioning on the result of the toss.

(c) Are \(X\) and \(Y\) uncorrelated?

(d) Are \(X\) and \(Y\) independent?

(e) Is the joint distribution of \(X\) and \(Y\) bivariate normal?

5. Normal Sample Mean and Sample Variance, Part 1

Let \(X_1, X_2, \ldots, X_n\) be i.i.d. with mean \(\mu\) and variance \(\sigma^2\). Let

\[ \bar{X} ~ = ~ \frac{1}{n} \sum_{i=1}^n X_i \]

denote the sample mean and

\[ S^2 ~=~ \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2 \]

denote the sample variance as defined earlier in the course.

(a) For \(1 \le i \le n\) let \(D_i = X_i - \bar{X}\). Find \(Cov(D_i, \bar{X})\).

(b) Now assume in addition that \(X_1, X_2, \ldots, X_n\) are i.i.d. normal \((\mu, \sigma^2)\). What is the joint distribution of \(\bar{X}, D_1, D_2, \ldots, D_{n-1}\)? Explain why \(D_n\) isn’t on the list.

(c) True or false (justify your answer): The sample mean and sample variance of an i.i.d. normal sample are independent of each other.

6. Normal Sample Mean and Sample Variance, Part 2

(a) Let \(R\) have the chi-squared distribution with \(n\) degrees of freedom. What is the mgf of \(R\)?

(b) For \(R\) as in Part (a), suppose \(R = V + W\) where \(V\) and \(W\) are independent and \(V\) has the chi-squared distribution with \(m < n\) degrees of freedom. Can you identify the distribution of \(W\)? Justify your answer.

(c) Let \(X_1, X_2, \ldots , X_n\) be any sequence of random variables and let \(\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i\). Let \(\alpha\) be any constant. Prove the sum of squares decomposition

\[ \sum_{i=1}^n (X_i - \alpha)^2 ~=~ \sum_{i=1}^n (X_i - \bar{X})^2 ~+~ n(\bar{X} - \alpha)^2. \]

(d) Now let \(X_1, X_2, \ldots, X_n\) be i.i.d. normal with mean \(\mu\) and variance \(\sigma^2 > 0\). Let \(S^2\) be the “sample variance” defined by

\[ S^2 ~=~ \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2. \]

Find a constant \(c\) such that \(cS^2\) has a chi-squared distribution. Provide the degrees of freedom.

[Use Parts (b) and (c) as well as the result of the previous exercise.]