Probability Distributions

Normal Distribution

参数: $\mu$ (均值), $\sigma^2$ (方差)

Probability density function (PDF):

$$f(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}}, \quad -\infty < x < \infty$$

Cumulative Distribution Function (CDF):

正态分布的CDF没有初等函数的封闭解析式。

$$F(x) = \Phi\left(\frac{x-\mu}{\sigma}\right) = \int_{-\infty}^{x} \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(t-\mu)^2}{2\sigma^2}} dt$$

Expectation:

$$E[X] = \mu$$

Variance:

$$\text{Var}(X) = \sigma^2$$

Moments:

$$E[(X-\mu)^n] = \begin{cases} 0 & \text{if } n \text{ is odd} \\ \sigma^n (n-1)!! & \text{if } n \text{ is even} \end{cases}$$

$(n-1)!! = (n-1)(n-3)\cdots 3 \cdot 1$ 是双阶乘

Moment generating function:

$$M_X(t) = e^{\mu t + \frac{1}{2}\sigma^2 t^2}$$

Proof: Using the substitution $y = \frac{x - \mu}{\sigma}$, so that $dx = \sigma \, dy$, we have:

$$\int_{\mathbb{R}} f(x) \, dx = \frac{1}{\sqrt{2\pi}\sigma} \int_{-\infty}^\infty e^{-\frac{(x - \mu)^2}{2\sigma^2}} \, dx = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy = \frac{1}{\sqrt{2\pi}} \cdot I,$$

where

$$I = \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy.$$

Note that this integral exists. Consider:

$$I^2 = \left( \int_{-\infty}^\infty e^{-\frac{x^2}{2}} \, dx \right) \left( \int_{-\infty}^\infty e^{-\frac{y^2}{2}} \, dy \right) = \int_{-\infty}^\infty \int_{-\infty}^\infty e^{-\frac{x^2 + y^2}{2}} \, dx \, dy.$$

We switch to polar coordinates, i.e., we substitute

$$x = r \cos \theta, \quad y = r \sin \theta, \quad r > 0, \; \theta \in [0, 2\pi).$$

We compute the Jacobian determinant:

$$\left| \frac{\partial(x, y)}{\partial(r, \theta)} \right| = \left| \begin{matrix} \cos\theta & -r \sin\theta \\ \sin\theta & r \cos\theta \end{matrix} \right| = r.$$

So we obtain:

$$I^2 = \int_0^\infty \int_0^{2\pi} e^{-\frac{r^2}{2}} r \, d\theta \, dr = 2\pi \int_0^\infty r e^{-\frac{r^2}{2}} \, dr.$$

Substitute $u = \frac{r^2}{2}$, so that $du = r \, dr$:

$$2\pi \int_0^\infty e^{-u} \, du = 2\pi [-e^{-u}]_0^\infty = 2\pi.$$

Hence, we conclude:

$$I = \sqrt{2\pi}, \quad \text{and therefore} \quad \int_{\mathbb{R}} f(x) \, dx = 1$$

.$\square$

Proof:

Expectation:

Substitute $z = \frac{x - \mu}{\sigma} \Rightarrow x = \sigma z + \mu$, and $dx = \sigma dz$:

$$\begin{aligned} \mathbb{E}[X] &= \int_{-\infty}^{\infty} (\sigma z + \mu) \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \sigma \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} z \cdot e^{-z^2/2} \, dz +\mu \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-z^2/2} \, dz \\ \end{aligned}$$

First notice that $z \cdot e^{-z^2/2}$ is an odd function. Since

$$\int_{-\infty}^{\infty} |z e^{-z^2/2}| \, dz = 2 \int_0^{\infty} z e^{-z^2/2} \, dz = 2 \cdot [-e^{-z^2/2}]_0^\infty = 1,$$

we have

$$\frac{\sigma}{\sqrt{2\pi}}\int_{-\infty}^{\infty} z \cdot e^{-z^2/2} \, dz = 0.$$

(Do keep in mind: if $f$ is an odd function, then

$$\int_{\infty}^{\infty}|f(x)|dx < \infty$$

is a necessary condition of $\int_{\infty}^{\infty}f(x)dx = 0$ . Because otherwise, $f(x)=x$ is a simple counterexample.)

Moreover, we have seen in the last proof that

$$\frac{1}{\sqrt{2\pi}} \int_{-\infty}^{\infty} e^{-z^2/2} \, dz = 1.$$

Therefore,

$$\mathbb{E}[X] = \sigma \cdot 0 + \mu \cdot 1 = \mu.$$

Variance:

We compute:

$$\mathrm{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 = \int_{-\infty}^{\infty} x^2 f(x) \, dx - \mu^2$$

Again substitute $z = \frac{x - \mu}{\sigma} \Rightarrow x = \sigma z + \mu$, and $dx = \sigma dz$:

$$\begin{aligned} \int_{-\infty}^{\infty} x^2 f(x) \, dx &= \int_{-\infty}^{\infty} (\sigma z + \mu)^2 \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \int_{-\infty}^{\infty} (\sigma^2 z^2 + 2\sigma \mu z + \mu^2) \cdot \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \, dz \\ &= \sigma^2 \int z^2 \phi(z) \, dz + 2\sigma\mu \int z \phi(z) \, dz + \mu^2 \int \phi(z) \, dz \\ &= \sigma^2 \cdot 1 + 2\sigma\mu \cdot 0 + \mu^2 \cdot 1 \\ &= \sigma^2 + \mu^2 \end{aligned}$$

,where

$$\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}.$$

In addition, since $\phi’(z) = z\phi(z)$,

$$\begin{aligned} \int_{-\infty}^{\infty} z^2 \phi(z) \, dz &= \int_{-\infty}^{\infty} z \cdot (-\phi'(z)) \, dz \\ &\overset{\text{integration by parts}}{=} [-z\phi(z)]_{-\infty}^{\infty} + \int_{-\infty}^{\infty} \phi(z) \, dz \\ &= 0+1 = 1. \end{aligned}$$

Finally, we have

$$\mathrm{Var}(X) = \int_{-\infty}^{\infty} x^2 f(x) \, dx - \mu^2 = \sigma^2 + \mu^2 - \mu^2 = \sigma^2$$

.$\square$

Bernoulli Distribution 伯努利分布

参数: $p$ (成功概率, $0 \le p \le 1$)

Probability mass function (PMF):

$$P(X=k) = p^k (1-p)^{1-k}, \quad k \in \{0, 1\}$$

Cumulative Distribution Function (CDF):

$$F(x) = \begin{cases} 0 & x < 0 \\ 1-p & 0 \le x < 1 \\ 1 & x \ge 1 \end{cases}$$

Expectation:

$$E[X] = p$$

Variance:

$$\text{Var}(X) = p(1-p)$$

Moments:

$$E[X^n] = p \quad (n \ge 1)$$

Moment generating function:

$$M_X(t) = 1 - p + pe^t$$

Binomial Distribution 二项分布

参数: $n$ (试验次数), $p$ (成功概率)

Probability mass function (PMF):

$$P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}, \quad k = 0, 1, \dots, n$$

Cumulative Distribution Function (CDF):

$$F(x) = \sum_{i=0}^{\lfloor x \rfloor} \binom{n}{i} p^i (1-p)^{n-i}$$

Expectation:

$$E[X] = np$$

Variance:

$$\text{Var}(X) = np(1-p)$$

Moments:

$$E[X^2] = n(n-1)p^2 + np$$

Moment generating function:

$$M_X(t) = (1 - p + pe^t)^n$$

Poisson Distribution 泊松分布

参数: $\lambda$ (单位时间/面积内的平均发生率, $\lambda > 0$)

Probability mass function (PMF):

$$P(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}, \quad k = 0, 1, 2, \dots$$

Cumulative Distribution Function (CDF):

$$F(x) = e^{-\lambda} \sum_{i=0}^{\lfloor x \rfloor} \frac{\lambda^i}{i!}$$

Expectation:

$$E[X] = \lambda$$

Variance:

$$\text{Var}(X) = \lambda$$

Moments:

$$E[X^2] = \lambda^2 + \lambda$$

Moment generating function:

$$M_X(t) = e^{\lambda(e^t - 1)}$$

Geometric Distribution 几何分布

参数: $p$ (成功概率)。此处定义为第 $k$ 次试验首次成功 (Support: $1, 2, \dots$)

Probability mass function (PMF):

$$P(X=k) = (1-p)^{k-1}p, \quad k = 1, 2, \dots$$

Cumulative Distribution Function (CDF):

$$F(x) = \begin{cases} 0 & x < 1 \\ 1 - (1-p)^{\lfloor x \rfloor} & x \ge 1 \end{cases}$$

Expectation:

$$E[X] = \frac{1}{p}$$

Variance:

$$\text{Var}(X) = \frac{1-p}{p^2}$$

Moments:

涉及到多重对数函数，无简单的初等函数通项。通常使用递推公式：

$$E[X^n] = \frac{1}{p} \sum_{k=0}^{n-1} \binom{n}{k} E[X^k] (1-p)$$ $$E[X^2] = \frac{2-p}{p^2}$$

Moment generating function:

$$M_X(t) = \frac{pe^t}{1 - (1-p)e^t}, \quad \text{for } t < -\ln(1-p)$$

Uniform Distribution 均匀分布

参数: $a, b$ (区间端点, $a < b$)

Probability density function (PDF):

$$f(x) = \begin{cases} \frac{1}{b-a} & a \le x \le b \\ 0 & \text{otherwise} \end{cases}$$

Cumulative Distribution Function (CDF):

$$F(x) = \begin{cases} 0 & x < a \\ \frac{x-a}{b-a} & a \le x < b \\ 1 & x \ge b \end{cases}$$

Expectation:

$$E[X] = \frac{a+b}{2}$$

Variance:

$$\text{Var}(X) = \frac{(b-a)^2}{12}$$

Moments:

$$E[X^n] = \frac{b^{n+1} - a^{n+1}}{(n+1)(b-a)}$$

Moment generating function:

$$M_X(t) = \frac{e^{tb} - e^{ta}}{t(b-a)}, \quad t \neq 0$$

Exponential Distribution 指数分布

参数: $\lambda$ (率参数, $\lambda > 0$)

Probability density function (PDF):

$$f(x) = \begin{cases} \lambda e^{-\lambda x} & x \ge 0 \\ 0 & x < 0 \end{cases}$$

Cumulative Distribution Function (CDF):

$$F(x) = \begin{cases} 1 - e^{-\lambda x} & x \ge 0 \\ 0 & x < 0 \end{cases}$$

Expectation:

$$E[X] = \frac{1}{\lambda}$$

Variance:

$$\text{Var}(X) = \frac{1}{\lambda^2}$$

Moments:

$$E[X^n] = \frac{n!}{\lambda^n}$$

Moment generating function:

$$M_X(t) = \frac{\lambda}{\lambda - t}, \quad t < \lambda$$

Gamma Distribution 伽玛分布

参数: $\alpha$ (形状参数 shape), $\beta$ (率参数 rate, sometimes $\theta = 1/\beta$ is used as scale)

Probability density function (PDF):

$$f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x > 0$$

Cumulative Distribution Function (CDF):

$$F(x) = \frac{\gamma(\alpha, \beta x)}{\Gamma(\alpha)}$$

其中 $\gamma(s, x) = \int_0^x t^{s-1} e^{-t} dt$。

Expectation:

$$E[X] = \frac{\alpha}{\beta}$$

Variance:

$$\text{Var}(X) = \frac{\alpha}{\beta^2}$$

Moments:

$$E[X^n] = \frac{\Gamma(\alpha+n)}{\beta^n \Gamma(\alpha)} = \frac{\alpha(\alpha+1)\cdots(\alpha+n-1)}{\beta^n}$$

Moment generating function:

$$M_X(t) = \left( \frac{\beta}{\beta - t} \right)^\alpha, \quad t < \beta$$

Chi-Squared Distribution 卡方分布

参数: $k$ (自由度 degrees of freedom, $k \in \mathbb{N}^*$)

卡方分布是伽玛分布的一个特例，其中$\alpha = k/2$, $\beta = 1/2$。

Probability density function (PDF):

$$f(x) = \frac{1}{2^{k/2}\Gamma(k/2)} x^{k/2 - 1} e^{-x/2}, \quad x > 0$$

Cumulative Distribution Function (CDF):

$$F(x) = \frac{\gamma(k/2, x/2)}{\Gamma(k/2)} = P\left(\frac{k}{2}, \frac{x}{2}\right)$$

其中 $\gamma(s, x)$ 为下不完全伽玛函数，$P$ 为正则化伽玛函数。

Expectation:

$$E[X] = k$$

Variance:

$$\text{Var}(X) = 2k$$

Moments:

$$E[X^n] = 2^n \frac{\Gamma(k/2 + n)}{\Gamma(k/2)} = \prod_{i=0}^{n-1} (k+2i) = k(k+2)(k+4)\cdots(k+2n-2)$$

Moment generating function:

$$M_X(t) = (1 - 2t)^{-k/2}, \quad t < \frac{1}{2}$$

Beta Distribution 贝塔分布

参数: $\alpha, \beta$ (形状参数 shape parameters, $\alpha > 0, \beta > 0$)

Probability density function (PDF):

$$f(x) = \frac{1}{B(\alpha, \beta)} x^{\alpha-1} (1-x)^{\beta-1}, \quad 0 < x < 1$$

其中 $B(\alpha, \beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$ 为贝塔函数。

Cumulative Distribution Function (CDF):

$$F(x) = I_x(\alpha, \beta) = \frac{B(x; \alpha, \beta)}{B(\alpha, \beta)}$$

其中 $B(x; \alpha, \beta) = \int_0^x t^{\alpha-1}(1-t)^{\beta-1} dt$ 为不完全贝塔函数 (Incomplete Beta Function)，$I_x$ 为正则化不完全贝塔函数。

Expectation:

$$E[X] = \frac{\alpha}{\alpha + \beta}$$

Variance:

$$\text{Var}(X) = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$$

Moments:

$$E[X^n] = \frac{B(\alpha+n, \beta)}{B(\alpha, \beta)} = \prod_{i=0}^{n-1} \frac{\alpha+i}{\alpha+\beta+i}$$

Moment generating function:

无初等函数形式，通常表示为合流超几何函数 (Confluent Hypergeometric Function):

$$M_X(t) = 1 + \sum_{k=1}^\infty \left( \prod_{r=0}^{k-1} \frac{\alpha+r}{\alpha+\beta+r} \right) \frac{t^k}{k!}$$

R

在 R 语言中，处理概率分布有一套非常标准的前缀命名规则。对于每种分布（比如 norm），都有 4 个核心函数：

• d (Density): 概率密度/质量函数 (PDF/PMF), e.g., dnorm
• p (Probability): 累积分布函数 (CDF), e.g., pnorm
• q (Quantile): 分位数函数 (CDF 的反函数), e.g., qnorm
• r (Random): 生成随机数, e.g., rnorm

一、离散型分布 (Discrete)

1. Bernoulli & Binomial (伯努利与二项分布)

R 中没有专门的 bernoulli 函数，伯努利分布就是 size = 1 的二项分布。

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

# 参数设定: n=10次试验, p=0.5成功率 n <- 10 p <- 0.5 # 1. PMF (Density): 恰好成功 5 次的概率 P(X=5) dbinom(x = 5, size = n, prob = p) # 2. CDF (Probability): 成功次数 <= 5 的概率 P(X<=5) pbinom(q = 5, size = n, prob = p) # 3. Quantile: 累积概率达到 0.95 时的成功次数 qbinom(p = 0.95, size = n, prob = p) # 4. Random: 模拟 5 组实验，每组试验 n 次 rbinom(n = 5, size = n, prob = p) # 特例：伯努利分布 (size = 1) rbinom(n = 10, size = 1, prob = 0.5) # 生成 10 个 0 或 1

2. Poisson Distribution (泊松分布)

1 2 3 4 5 6 7 8 9 10 11

# 参数设定: lambda = 4 lam <- 4 # 1. PMF: 发生 3 次的概率 P(X=3) dpois(x = 3, lambda = lam) # 2. CDF: 发生次数 <= 3 的概率 P(X<=3) ppois(q = 3, lambda = lam) # 3. Random: 生成 10 个符合泊松分布的随机数 rpois(n = 10, lambda = lam)

3. Geometric Distribution (几何分布)

R 中的几何分布定义为失败次数 ($X \in \{0, 1, \dots\}$)，直到第一次成功。

• 如果使用的是 $X \in \{1, 2, \dots\}$ (第 $k$ 次首次成功)，则 R 中的 x 应该是 k-1。

1 2 3 4 5 6 7 8 9 10 11 12

# 参数: p = 0.2 prob <- 0.2 # 1. PMF: 在第 5 次试验首次成功 (意味着前 4 次失败) # 公式: P(X=5 in math definition) -> dgeom(4, p) dgeom(x = 4, prob = prob) # 2. CDF: 在第 5 次及以前成功的概率 pgeom(q = 4, prob = prob) # 3. Random: 生成 10 个随机样本 (返回的是失败次数) rgeom(n = 10, prob = prob)

---

二、连续型分布 (Continuous)

4. Uniform Distribution (均匀分布)

1 2 3 4 5 6 7 8 9 10 11 12

# 参数: 区间 [a, b] -> min=0, max=10 a <- 0 b <- 10 # 1. PDF: 在 x=5 处的密度 (对于均匀分布是常数 1/(b-a)) dunif(x = 5, min = a, max = b) # 2. CDF: P(X <= 5) punif(q = 5, min = a, max = b) # 3. Random: 生成 5 个 [0, 10] 之间的随机数 runif(n = 5, min = a, max = b)

5. Exponential Distribution (指数分布)

1 2 3 4 5 6 7 8 9 10 11

# 参数: lambda (rate) = 0.5 lam <- 0.5 # 1. PDF: f(x) at x=2 dexp(x = 2, rate = lam) # 2. CDF: P(X <= 2) = 1 - e^(-lambda * 2) pexp(q = 2, rate = lam) # 3. Random: 生成符合指数分布的随机数 rexp(n = 5, rate = lam)

6. Normal Distribution (正态分布)

注意：R 使用标准差 sd ($\sigma$) 作为参数，而不是方差 ($\sigma^2$)。

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# 参数: mean=0, sd=1 (标准正态) mu <- 0 sigma <- 1 # 1. PDF: f(x) dnorm(x = 1.96, mean = mu, sd = sigma) # 2. CDF: P(X <= 1.96) ≈ 0.975 pnorm(q = 1.96, mean = mu, sd = sigma) # 3. Quantile: 查表逆运算，P(X <= z) = 0.975，求 z qnorm(p = 0.975, mean = mu, sd = sigma) # 结果约为 1.96 # 4. Random: 生成正态分布数据 rnorm(n = 10, mean = mu, sd = sigma)

7. Gamma Distribution (伽玛分布)

R 支持 rate ($\beta$) 或 scale ($1/\beta$) 参数。

1 2 3 4 5 6 7 8 9 10 11 12

# 参数: alpha (shape) = 2, beta (rate) = 0.5 alpha <- 2 beta <- 0.5 # 1. PDF: f(x) dgamma(x = 4, shape = alpha, rate = beta) # 2. CDF: F(x) pgamma(q = 4, shape = alpha, rate = beta) # 3. Random rgamma(n = 10, shape = alpha, rate = beta)