Lebesgue Decomposition of Probability Distributions II (The CDF Approach)

Types of Distributions.

Definition (Absolutely Continuous Distribution). A cumulative distribution function (CDF) $F(x)$ represents an absolutely continuous distribution if there exists a non-negative, integrable function $f(x)$, called the probability density function (PDF), such that for all $x\in \mathbb{R}$,

$$F(x)={\int }_{-\infty }^{x}f(t)dt$$

An absolutely continuous CDF is continuous everywhere and differentiable almost everywhere, with its derivative being the PDF, $F^{\prime }(x)=f(x)$ a.e.

Definition (Discrete Distribution). A CDF $F(x)$ represents a discrete distribution if it is a step function that increases only by a countable number of finite jumps. If the jump points are the countable set $S=\{x_1,x_2,\dots \}$, then $F(x)$ can be written as

$$F(x)=\sum _{x_k\in S,x_k\le x}{p}_k$$

where $p_k=F(x_k)-{\lim }_{y\to x_k^{-}}F(y)$ is the jump size at $x_k$, and ${\sum }_k{p}_k=1$.

Definition (Singular Continuous Distribution). A CDF $F(x)$ represents a singular continuous distribution if it satisfies two conditions:

1. $F(x)$ is continuous for all $x\in \mathbb{R}$.
2. Its derivative exists and is zero almost everywhere, i.e., $F^{\prime }(x)=0$ for all $x$ except for a set of Lebesgue measure zero.

The Cantor function is the canonical example of a singular continuous CDF.

The Lebesgue Decomposition Theorem.

Theorem (Lebesgue Decomposition for CDFs). Any cumulative distribution function $F(x)$ can be uniquely decomposed into a convex combination

$$F(x)={\alpha }_1{F}_{ac}(x)+{\alpha }_2{F}_d(x)+{\alpha }_3{F}_{sc}(x)$$

where $F_{ac}$ is an absolutely continuous CDF, $F_d$ is a discrete CDF, and $F_{sc}$ is a singular continuous CDF. The coefficients ${\alpha }_1,{\alpha }_2,{\alpha }_3$ are non-negative and sum to 1.

Proof. Existence. Let $F(x)$ be an arbitrary CDF. The proof proceeds by construction in two steps.

1. Extracting the Discrete Part. Let $D=\{x_1,x_2,\dots \}$ be the set of discontinuity points of $F$. This set is countable. For each $x_k\in D$, define the jump size $p_k=F(x_k)-{\lim }_{y\to x_k^{-}}F(y)$. Construct the function $F_d^{\prime }(x)={\sum }_{x_k\in D,x_k\le x}{p}_k$. Let ${\alpha }_2={\lim }_{x\to \infty }{F}_d^{\prime }(x)={\sum }_k{p}_k$. If ${\alpha }_2>0$, define the discrete CDF $F_d(x)=\frac{1}{{\alpha }_2}{F}_d^{\prime }(x)$. If ${\alpha }_2=0$, the distribution has no discrete part. Now define the continuous part of $F$ as $F_c(x)=F(x)-F_d^{\prime }(x)$. The function $F_c(x)$ is continuous because all jumps of $F$ have been subtracted out. It is also non-decreasing.

2. Decomposing the Continuous Part. Any non-decreasing function like $F_c(x)$ is differentiable almost everywhere. Let its derivative be $f_c(x)=F_c^{\prime }(x)$. The function $f_c(x)$ is non-negative and integrable. We can now define the absolutely continuous component of $F_c(x)$ by integrating its derivative:

   $$F_{ac}^{\prime }(x)={\int }_{-\infty }^x{f}_c(t)dt={\int }_{-\infty }^x{F}_c^{\prime }(t)dt$$

   Next, define the singular continuous component as the remainder: $F_{sc}^{\prime }(x)=F_c(x)-F_{ac}^{\prime }(x)$. By construction, $F_{sc}^{\prime }(x)$ is continuous. Its derivative is $F_{sc}^{\prime }{}^{\prime }(x)=F_c^{\prime }(x)-F_{ac}^{\prime }{}^{\prime }(x)=f_c(x)-f_c(x)=0$ almost everywhere. Thus, $F_{sc}^{\prime }(x)$ corresponds to a singular continuous distribution.

   Let ${\alpha }_1={\lim }_{x\to \infty }{F}_{ac}^{\prime }(x)$ and ${\alpha }_3={\lim }_{x\to \infty }{F}_{sc}^{\prime }(x)$. If ${\alpha }_1>0$, define $F_{ac}(x)=\frac{1}{{\alpha }_1}{F}_{ac}^{\prime }(x)$. If ${\alpha }_3>0$, define $F_{sc}(x)=\frac{1}{{\alpha }_3}{F}_{sc}^{\prime }(x)$. We have constructed the decomposition $F(x)=F_d^{\prime }(x)+F_{ac}^{\prime }(x)+F_{sc}^{\prime }(x)={\alpha }_2{F}_d(x)+{\alpha }_1{F}_{ac}(x)+{\alpha }_3{F}_{sc}(x)$. The coefficients sum to one:

   $${\alpha }_1+{\alpha }_2+{\alpha }_3=\underset{x\to \infty }{\lim }(F_{ac}^{\prime }(x)+F_d^{\prime }(x)+F_{sc}^{\prime }(x))=\underset{x\to \infty }{\lim }F(x)=1$$

This establishes the existence of the decomposition.

Uniqueness. Suppose $F(x)={\alpha }_1{F}_1+{\alpha }_2{F}_2+{\alpha }_3{F}_3$ and $F(x)={\beta }_1{G}_1+{\beta }_2{G}_2+{\beta }_3{G}_3$ are two such decompositions. The set of discontinuities of $F$ and the corresponding jump sizes are uniquely determined by $F$. Only the discrete components contribute to jumps. Thus, the function representing all jumps, ${\alpha }_2{F}_2(x)$, must be identical to ${\beta }_2{G}_2(x)$. Taking the limit as $x\to \infty$ gives ${\alpha }_2={\beta }_2$. If ${\alpha }_2>0$, then $F_2=G_2$.

This implies the continuous parts are equal: $F_c(x)={\alpha }_1{F}_1+{\alpha }_3{F}_3={\beta }_1{G}_1+{\beta }_3{G}_3$. Differentiating this equation gives $F_c^{\prime }(x)={\alpha }_1{F}_1^{\prime }(x)+{\alpha }_3{F}_3^{\prime }(x)={\beta }_1{G}_1^{\prime }(x)+{\beta }_3{G}_3^{\prime }(x)$. Since $F_3$ and $G_3$ are singular continuous, $F_3^{\prime }(x)=G_3^{\prime }(x)=0$ almost everywhere. Therefore, ${\alpha }_1{F}_1^{\prime }(x)={\beta }_1{G}_1^{\prime }(x)$ almost everywhere. The absolutely continuous part of a function is uniquely determined by integrating its derivative, so ${\alpha }_1{F}_1(x)={\int }_{-\infty }^x{\alpha }_1{F}_1^{\prime }(t)dt={\int }_{-\infty }^x{\beta }_1{G}_1^{\prime }(t)dt={\beta }_1{G}_1(x)$. Taking the limit as $x\to \infty$ gives ${\alpha }_1={\beta }_1$. If ${\alpha }_1>0$, then $F_1=G_1$. By subtraction, it follows that ${\alpha }_3{F}_3={\beta }_3{G}_3$, which implies ${\alpha }_3={\beta }_3$ and $F_3=G_3$. The decomposition is unique. $\square$```