Probability Theory (Graduate)

Graduate probability theory is the study of randomness built on a rigorous mathematical foundation. Instead of treating probabilities as intuitive fractions or long-run frequencies, the graduate approach formalizes random phenomena using measure theory, then develops powerful tools to analyze convergence, dependence, and limiting behavior. This framework underpins modern statistics, stochastic processes, information theory, and much of quantitative economics and finance.

What follows is a structured tour of core topics: measure-theoretic probability, modes of convergence, characteristic functions, martingales, and central limit theorems. The goal is not only to state theorems, but to explain what they accomplish and why they are indispensable.

Measure-theoretic foundations

At the graduate level, probability starts with a probability space $(\Omega ,\mathcal{F},\mathbb{P})$:

• $\Omega$ is the sample space of outcomes.
• $\mathcal{F}$ is a $\sigma$-algebra of events (the measurable subsets of $\Omega$).
• $\mathbb{P}$ is a probability measure with $\mathbb{P}(\Omega )=1$.

This setup matters because many random variables of interest live on complicated spaces (paths of a process, function spaces, infinite sequences). Measure theory provides the correct language for defining probability in these settings without contradictions.

A random variable is a measurable map $X:(\Omega ,\mathcal{F})\to (\mathbb{R},\mathcal{B})$, where $\mathcal{B}$ is the Borel $\sigma$-algebra. Measurability ensures events like $\{X\le t\}$ belong to $\mathcal{F}$, so probabilities like $\mathbb{P}(X\le t)$ are well-defined.

Expectation as an integral

Expectation is defined as a Lebesgue integral: \[ \mathbb{E}[X] = \int_{\Omega} X(\omega)\, d\mathbb{P}(\omega), \] when integrability conditions are satisfied. This view unifies discrete and continuous cases and provides precise tools for interchanging limits and integrals, which is central to convergence and limit theorems.

Two indispensable results are:

• Monotone Convergence Theorem (MCT): for $0\le X_n\uparrow X$, $\mathbb{E}[X_n]\uparrow \mathbb{E}[X]$.
• Dominated Convergence Theorem (DCT): if $X_n\to X$ a.s. and $|X_n|\le Y$ with $\mathbb{E}[Y]<\infty$, then $\mathbb{E}[X_n]\to \mathbb{E}[X]$.

These are not technical luxuries. They are what make it possible to justify passing limits through expectations when proving laws of large numbers or central limit theorems.

Conditional expectation

Conditional expectation is treated as a random variable rather than a number. For an integrable $X$ and a sub-$\sigma$-algebra $\mathcal{G}\subseteq \mathcal{F}$, the conditional expectation $\mathbb{E}[X\mid \mathcal{G}]$ is the $\mathcal{G}$-measurable random variable satisfying \[ \intG \mathbb{E}[X\mid \mathcal{G}]\, d\mathbb{P} = \intG X\, d\mathbb{P} \quad \text{for all } G\in\mathcal{G}. \] This definition is the gateway to martingales and modern stochastic analysis, because it formalizes “best prediction given current information.”

Modes of convergence

Probability theory distinguishes several notions of convergence because random variables can converge in different senses, with different implications.

Almost sure, in probability, and in $L^p$

Common modes include:

• Almost sure convergence: $X_n\to X$ a.s. if $\mathbb{P}(\{\omega :X_n(\omega )\to X(\omega )\})=1$.
• Convergence in probability: $X_n\to X$ in probability if for all $\epsilon >0$, $\mathbb{P}(|X_n-X|>\epsilon )\to 0$.
• __MATH_INLINE_30__ convergence: $X_n\to X$ in $L^p$ if $\mathbb{E}[|X_n-X{|}^p]\to 0$.

A typical hierarchy is: \[ Xn \to X \text{ in } L^p \Rightarrow Xn \to X \text{ in probability}, \] and almost sure convergence implies convergence in probability, but not vice versa in general.

These distinctions are not pedantry. For example, laws of large numbers can often be proved in probability first, then strengthened to almost sure convergence under additional conditions.

Weak convergence and distributional limits

A separate concept is convergence in distribution (weak convergence), written $X_n\Rightarrow X$. It means the distribution functions converge at continuity points of the limit. Weak convergence is the natural framework for central limit theorems, because the normalized sums often do not converge pointwise or in $L^p$, but their distributions stabilize.

A key practical question in graduate probability is: Which convergence is needed for the result I want? In statistics, convergence in distribution supports asymptotic approximations; in stochastic processes, almost sure statements can be crucial for pathwise properties.

Characteristic functions

A characteristic function is the Fourier transform of a probability distribution: \[ \varphi_X(t) = \mathbb{E}[e^{itX}], \quad t\in \mathbb{R}. \] Characteristic functions always exist (unlike moment generating functions) and encode the full distribution. Two cornerstone facts:

1. Uniqueness: ${\phi }_X={\phi }_Y$ implies $X$ and $Y$ have the same distribution.
2. Continuity theorem: ${\phi }_{X_n}(t)\to \phi (t)$ pointwise with $\phi$ continuous at $0$ implies $X_n\Rightarrow X$ for some $X$ with characteristic function $\phi$.

Characteristic functions are especially effective for sums of independent random variables since \[ \varphi{X+Y}(t) = \varphiX(t)\varphiY(t) \] when MATHINLINE45 and MATHINLINE46_ are independent. This multiplicative structure turns limit theorems into analytic problems about convergence of functions.

Martingales and information over time

A martingale formalizes a fair game. Let $({\mathcal{F}}_n)$ be a filtration representing information revealed over time. A process $(M_n)$ is a martingale if:

• $M_n$ is ${\mathcal{F}}_n$-measurable,
• $\mathbb{E}[|M_n|]<\infty$,
• $\mathbb{E}[M_{n+1}\mid {\mathcal{F}}_n]=M_n$.

Martingales show up naturally by taking conditional expectations, constructing partial sums of mean-zero increments, or modeling price processes under “no arbitrage” assumptions in finance.

Martingale inequalities and stopping

Martingale methods provide robust bounds and convergence results. Doob-type inequalities bound maximal fluctuations, which is crucial for proving almost sure convergence. Optional stopping results clarify when a stopped martingale preserves expectation, and when it fails due to integrability or unbounded stopping times.

One of the most useful qualitative conclusions is that martingale structure often yields convergence without independence, making it a powerful generalization of classical tools.

Limit theorems: laws of large numbers and central limit behavior

Limit theorems explain how complex random systems exhibit regular behavior at scale.

Laws of large numbers

A typical law of large numbers states that for i.i.d. $X_1,X_2,\dots$ with $\mathbb{E}[X_1]=\mu$, \[ \frac{1}{n}\sum{k=1}^n Xk \to \mu, \] in an appropriate sense (often almost surely in the strong law, in probability in the weak law). The measure-theoretic toolkit (MCT, DCT, truncation arguments) and concentration inequalities are standard techniques for proving these results under minimal assumptions.

Central limit theorems (CLTs)

The central limit theorem explains why normal distributions appear ubiquitously. In its classical form, for i.i.d. variables with mean $\mu$ and variance ${\sigma }^2\in (0,\infty )$, \[ \frac{\sum{k=1}^n Xk - n\mu}{\sigma\sqrt{n}} \Rightarrow Z, \] where $Z$ is standard normal.

At the graduate level, the CLT is not a single theorem but a family of results under varying assumptions. One learns to handle:

• Triangular arrays (non-identically distributed summands).
• Lindeberg-type conditions controlling the contribution of large summands.
• Lyapunov-type conditions formulated via moments.

Characteristic functions provide a clean route to CLTs, but martingale CLTs and other approaches extend the theory to dependent data, which matters in time series, random walks, and stochastic algorithms.

How the pieces fit together

Measure-theoretic probability supplies the