问题及解答

Definition (随机变量的线性组合)

Given a collection of $n$ random variables $X_1,\ldots,X_n$ and $n$ numerical constants $a_1,\ldots,a_n$, the rv
\[
Y=a_1 X_1+\cdots+a_n X_n=\sum_{i=1}^{n}a_i X_i
\]
is called a linear combination of the $X_i$'s.
 

Prove the following proposition.

Prop 1. Let $X_1,X_2,\ldots,X_n$ have mean values $\mu_1,\ldots,\mu_n$, respectively, and variances of $\sigma_1^2,\ldots,\sigma_n^2$, respectively.

(1) Whether or not the $X_i$'s are independent,
  \[
  \begin{split}
  E(a_1 X_1+a_2 X_2+\cdots+a_n X_n)&=a_1 E(X_1)+a_2 E(X_2)+\cdots+a_n E(X_n)\\
  &=a_1\mu_1+a_2\mu_2+\cdots+a_n\mu_n
  \end{split}
  \]

(2) If $X_1,\ldots,X_n$ are independent,
  \[
  \begin{split}
  V(a_1 X_1+a_2 X_2+\cdots+a_n X_n)&=a_1^2 V(X_1)+a_2^2 V(X_2)+\cdots+a_n^2 V(X_n)\\
  &=a_1^2\sigma_1^2+a_2^2\sigma_2^2+\cdots+a_n^2\sigma_n^2
  \end{split}
  \]
  and
  \[
  \sigma_{a_1 X_1+\cdots+a_n X_n}=\sqrt{a_1^2\sigma_1^2+\cdots+a_n^2\sigma_n^2}
  \]

(3) For any $X_1,\ldots,X_n$,
  \[
  V(a_1 X_1+\cdots+a_n X_n)=\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j\mathrm{Cov}(X_i,X_j)
  \]

 

Then, we get the following proposition.

Prop 2. Let $X_1,X_2,\ldots,X_n$ be a random sample from a distribution with mean value $\mu$ and standard deviation $\sigma$. Then

• $E(\bar{X})=\mu_{\bar{X}}=\mu$.
• $V(\bar{X})=\sigma_{\bar{X}}^2=\frac{\sigma^2}{n}$ and $\sigma_{\bar{X}}=\frac{\sigma}{\sqrt{n}}$.

Here $\bar{X}=\frac{1}{n}(X_1+X_2+\cdots+X_n)$.

In addition, with $T_o=X_1+\cdots+X_n$ (the sample total), $E(T_o)=n\mu$, $V(T_o)=n\sigma^2$, and $\sigma_{T_o}=\sqrt{n}\sigma$.

注: 有时也记 $D(X)=V(X)$.

样本方差 $s^2$ 定义为

\[
s^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2
\]

其开方 $s$ 称为样本均方差或样本标准差.

\[
s=\sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2}
\]

样本 $k$ 阶原点矩 $m_k=\frac{1}{n}\sum_{i=1}^{n}X_i^k$, $k=1,2,\ldots$

样本 $k$ 阶中心矩 $M_k=\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^k$, $k=1,2,\ldots$

Prop. $E(s^2)=D(X)=\sigma^2$.

对于 Prop 1.

(1) 显然我们可以使用归纳法证明

\[
E(\sum_{i=1}^{n}a_i X_i)=\sum_{i=1}^{n}a_i E(X_i).
\]

也就是说只需对 $n=2$ 证明即可.

\[
\begin{split}
E(a_1 X_1+a_2 X_2)&=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}(ax_1+ax_2)f(x_1,x_2)dx_1 dx_2\\
&=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}ax_1\cdot f(x_1,x_2)dx_1 dx_2+\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}ax_2\cdot f(x_1,x_2)dx_1 dx_2\\
&=a_1\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}x_1f(x_1,x_2)dx_1 dx_2+a_2\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}x_2f(x_1,x_2)dx_1 dx_2\\
&=a_1 E(X_1)+a_2 E(X_2).
\end{split}
\]

(2)

$X_1,X_2,\ldots,X_n$ 独立, 指 $f(x_1,x_2,\ldots,x_n)=f_{X_1}(x_1)\cdot f_{X_2}(x_2)\cdots f_{X_n}(x_n)$.

仍然, 我们先考虑 $n=2$ 的情形. $X_1$ 与 $X_2$ 独立, 等价于 $E(X_1 X_2)=E(X_1)\cdot E(X_2)$.

\[
\begin{split}
V(a_1 X_1+a_2 X_2)&=E\bigl((a_1 X_1+a_2 X_2)^2\bigr)-\Bigl(E(a_1 X_1+a_2 X_2)\Bigr)^2\\
&=E\bigl(a_1^2 X_1^2+2a_1 a_2 X_1 X_2+a_2^2 X_2^2\bigr)-\Bigl(a_1 E(X_1)+a_2 E(X_2)\Bigr)^2\\
&=a_1^2 E(X_1^2)+2a_1 a_2E(X_1 X_2)+a_2^2 E(X_2^2)-a_1^2\bigl(E(X_1)\bigr)^2-2a_1 a_2 E(X_1)E(X_2)-a_2^2\bigl(E(X_2)\bigr)^2\\
&=a_1^2\Bigl[E(X_1^2)-\bigl(E(X_1)\bigr)^2\Bigr]+a_2^2\Bigl[E(X_2^2)-\bigl(E(X_2)\bigr)^2\Bigr]\\
&=a_1^2 V(X_1)+a_2^2 V(X_2)
\end{split}
\]

这里倒数第二个等号用到了 $E(X_1 X_2)=E(X_1)E(X_2)$, 即 $X_1$ 与 $X_2$ 独立的条件.

假设对于 $n=k$ 时, 成立

\[
V(a_1 X_1+\cdots +a_k X_k)=a_1^2 V(X_1)+\cdots+a_k^2 V(X_k)
\]

则当 $n=k+1$ 时, 令 $Y=a_1 X_1+\cdots +a_k X_k$, 

\[
\begin{split}
V(a_1 X_1+\cdots +a_k X_k+a_{k+1}X_{k+1})&=V(Y+a_{k+1}X_{k+1})\\
&=1^2 V(Y)+a_{k+1}^2 V(X_{k+1})\\
&=V(Y)+a_{k+1}^2 V(X_{k+1})\\
\end{split}
\]

根据归纳假设, $V(Y)=a_1^2 V(X_1)+\cdots+a_k^2 V(X_k)$, 故

\[
V(a_1 X_1+\cdots +a_k X_k+a_{k+1}X_{k+1})=a_1^2 V(X_1)+\cdots+a_k^2 V(X_k)+a_{k+1}^2 V(X_{k+1}).
\]

得证.

(c)

\[
\begin{split}
V(a_1 X_1+\cdots+a_n X_n)&=E\Bigl[(a_1 X_1+\cdots+a_n X_n)^2\Bigr]-\bigl[E(a_1 X_1+\cdots+a_n X_n)\bigr]^2\\
&=E(\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j X_i X_j)-\biggl(\sum_{i=1}^{n}a_i E(X_i)\biggr)^2\\
&=\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j E(X_i X_j)-\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j E(X_i)E(X_j)\\
&=\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j\Bigl[E(X_i X_j)-E(X_i)E(X_j)\Bigr]\\ 
&=\sum_{i=1}^{n}\sum_{j=1}^{n}a_i a_j\mathrm{Cov}(X_i, X_j).
\end{split}
\]

Prop 2. 是 Prop 1. 的特殊情形.

\[
\begin{split}
E(s^2)&=E\Bigl(\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2\Bigr)\\
&=\frac{1}{n-1}E\Bigl(\sum_{i=1}^{n}(X_i-\bar{X})^2\Bigr)\\
&=\frac{1}{n-1}E\Bigl(\sum_{i=1}^{n}(X_i^2-2X_i\cdot\bar{X}+\bar{X}^2)\Bigr)\\
&=\frac{1}{n-1}E\Bigl[\sum_{i=1}^{n}X_i^2-2\bar{X}\cdot\sum_{i=1}^{n}X_i+\sum_{i=1}^{n}\bar{X}^2\Bigr]\\
&=\frac{1}{n-1}E\Bigl[\sum_{i=1}^{n}X_i^2-2\bar{X}\cdot n\bar{X}+n\bar{X}^2\Bigr]\\
&=\frac{1}{n-1}E\Bigl[\sum_{i=1}^{n}X_i^2-n\bar{X}^2\Bigr]\\
&=\frac{1}{n-1}\Bigl[\sum_{i=1}^{n}E(X_i^2)-nE(\bar{X}^2)\Bigr]
\end{split}
\]

注意到 $D(X)=E(X^2)-(E(X))^2$, 故 $E(X^2)=D(X)+(E(X))^2=\sigma^2+\mu^2$. 因此这里

\[
E(X_i^2)=\sigma^2+\mu^2
\]

另一方面, 

\[
E(\bar{X}^2)=D(\bar{X})+(E(\bar{X}))^2
\]

再由 Prop 2, 可得

\[
E(\bar{X}^2)=D(\bar{X})+(E(\bar{X}))^2=\frac{\sigma^2}{n}+\mu^2.
\]

因此, 

\[
\begin{split}
E(s^2)&=\frac{1}{n-1}\Bigl[n(\sigma^2+\mu^2)-n\cdot E(\bar{X}^2)\Bigr]\\
&=\frac{1}{n-1}\Bigl[n(\sigma^2+\mu^2)-n\cdot(\frac{\sigma^2}{n}+\mu^2)\Bigr]\\
&=\frac{1}{n-1}\Bigl[n\sigma^2+n\mu^2-\sigma^2-n\mu^2\Bigr]\\
&=\frac{1}{n-1}\cdot(n-1)\sigma^2\\
&=\sigma^2.
\end{split}
\]

得证.