Applications in Probability Theory

The Joint Probability Density

A Two-Dimensional Continuous Random Variable

Joint probability density

Suppose (X,Y)(X, Y) is a two-dimensional continuous random variable. If there is a nonnegative integrable function f(x,y)f(x,y) such that for any plane region DD,

P{(X,Y)∈D}=∬Df(x,y) dx dyP\{(X,Y) \in D\} = \iint_D f(x,y) \, dx\,dy

then f(x,y)f(x,y) is called the joint probability density function of (X,Y)(X,Y) (or simply the joint density).

The joint density satisfies:

  1. Nonnegativity: f(x,y)≥0f(x,y) \geq 0
  2. Normalisation: ∬R2f(x,y) dx dy=1\iint_{\mathbb{R}^2} f(x,y) \, dx\,dy = 1

The Geometric Meaning of a Probability

P{(X,Y)∈D}P\{(X,Y) \in D\} equals the volume of the cylinder with curved top whose base is the region DD and whose top is the surface z=f(x,y)z = f(x,y).

The Marginal Probability Density

Marginal probability density

From the joint density f(x,y)f(x,y) you can obtain the probability density of XX and of YY themselves (the marginal densities):

fX(x)=∫−∞+∞f(x,y) dyf_X(x) = \int_{-\infty}^{+\infty} f(x,y) \, dy fY(y)=∫−∞+∞f(x,y) dxf_Y(y) = \int_{-\infty}^{+\infty} f(x,y) \, dx

The marginal density is the integral of the joint density over the other variable — it “compresses” a two-dimensional distribution into one dimension.

The Conditional Probability Density

Conditional probability density

When fY(y)>0f_Y(y) > 0, the conditional probability density of XX given Y=yY = y is:

fX∣Y(x∣y)=f(x,y)fY(y)f_{X|Y}(x|y) = \frac{f(x,y)}{f_Y(y)} When fX(x)>0f_X(x) > 0, the conditional probability density of YY given X=xX = x is: fY∣X(y∣x)=f(x,y)fX(x)f_{Y|X}(y|x) = \frac{f(x,y)}{f_X(x)}

Independence

Theorem1

The necessary and sufficient condition for the random variables XX and YY to be mutually independent is:

f(x,y)=fX(x)⋅fY(y)f(x,y) = f_X(x) \cdot f_Y(y)

for almost every (x,y)(x,y).

The Mathematical Expectation

The Expectation of a Function of a Two-Dimensional Random Variable

The expectation of a function of a random variable

Suppose Z=g(X,Y)Z = g(X,Y) is a function of a two-dimensional random variable. Then:

E(Z)=E[g(X,Y)]=∬R2g(x,y)f(x,y) dx dyE(Z) = E[g(X,Y)] = \iint_{\mathbb{R}^2} g(x,y) f(x,y) \, dx\,dy

In particular:

  • E(X)=∬xf(x,y) dx dy=∫xfX(x) dxE(X) = \iint x f(x,y) \, dx\,dy = \int x f_X(x) \, dx
  • E(Y)=∬yf(x,y) dx dy=∫yfY(y) dyE(Y) = \iint y f(x,y) \, dx\,dy = \int y f_Y(y) \, dy

The Covariance and the Correlation Coefficient

The covariance and the correlation coefficient

The covariance: Cov(X,Y)=E[(X−EX)(Y−EY)]=E(XY)−EX⋅EY\text{Cov}(X,Y) = E[(X-EX)(Y-EY)] = E(XY) - EX \cdot EY =∬(x−EX)(y−EY)f(x,y) dx dy= \iint (x-EX)(y-EY) f(x,y) \, dx\,dy

The correlation coefficient: ρXY=Cov(X,Y)D(X)D(Y)\rho_{XY} = \frac{\text{Cov}(X,Y)}{\sqrt{D(X)}\sqrt{D(Y)}}

If X,YX, Y are independent, then Cov(X,Y)=0\text{Cov}(X,Y) = 0 (the converse does not hold).

Worked Examples

Example 1: The Normalisation and the Probabilities of a Joint Density

Suppose the joint density of a two-dimensional random variable (X,Y)(X,Y) is:

f(x,y)={Axy,0≤x≤1, 0≤y≤10,otherwisef(x,y) = \begin{cases} Axy, & 0 \leq x \leq 1, \, 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}

Find: (1) the constant AA; (2) P{X+Y≤1}P\{X+Y \leq 1\}; (3) the marginal densities fX(x),fY(y)f_X(x), f_Y(y).

the joint densitynormalisationthe marginal densitycomputing a probability

Approach:

Use the normalisation to find AA, a double integral to find the probability, and a single integral to find the marginal densities.

Steps:

(1) The normalisation: ∫01∫01Axy dx dy=A∫01x dx∫01y dy=A⋅12⋅12=A4=1\int_0^1 \int_0^1 Axy \, dx\,dy = A \int_0^1 x \, dx \int_0^1 y \, dy = A \cdot \frac{1}{2} \cdot \frac{1}{2} = \frac{A}{4} = 1 so A=4A = 4.

(2) P{X+Y≤1}P\{X+Y \leq 1\}: The region D:0≤x≤1, 0≤y≤1−xD: 0 \leq x \leq 1, \, 0 \leq y \leq 1-x P=∫01dx∫01−x4xy dy=∫014x⋅(1−x)22dx=2∫01x(1−2x+x2)dx=2∫01(x−2x2+x3)dx=2[x22−2x33+x44]01=2(12−23+14)=2⋅112=16P = \int_0^1 dx \int_0^{1-x} 4xy \, dy = \int_0^1 4x \cdot \frac{(1-x)^2}{2} dx = 2\int_0^1 x(1-2x+x^2) dx = 2\int_0^1 (x-2x^2+x^3) dx = 2[\frac{x^2}{2} - \frac{2x^3}{3} + \frac{x^4}{4}]_0^1 = 2(\frac{1}{2} - \frac{2}{3} + \frac{1}{4}) = 2 \cdot \frac{1}{12} = \frac{1}{6}

(3) The marginal densities: fX(x)=∫014xy dy=4x⋅12=2xf_X(x) = \int_0^1 4xy \, dy = 4x \cdot \frac{1}{2} = 2x (for 0≤x≤10 \leq x \leq 1), and 0 otherwise. fY(y)=∫014xy dx=4y⋅12=2yf_Y(y) = \int_0^1 4xy \, dx = 4y \cdot \frac{1}{2} = 2y (for 0≤y≤10 \leq y \leq 1), and 0 otherwise.

Answer: (1) A=4A=4; (2) P{X+Y≤1}=16P\{X+Y\leq1\} = \frac{1}{6}; (3) fX(x)=2x, fY(y)=2yf_X(x)=2x, \, f_Y(y)=2y (for 0≤x,y≤10\leq x,y\leq1)

Example 2: A Uniform Distribution and the Expectation

Suppose (X,Y)(X,Y) is uniformly distributed over the region D:0≤x≤1, x2≤y≤xD: 0 \leq x \leq 1, \, x^2 \leq y \leq x. Find E(X)E(X) and E(Y)E(Y).

a uniform distributionthe mathematical expectationthe joint densitya double integral

Approach:

First find the area of the region to get the joint density, and then use the double integral to find the expectation.

Steps:

  1. The area of the region: S=∫01dx∫x2xdy=∫01(x−x2)dx=[x22−x33]01=16S = \int_0^1 dx \int_{x^2}^x dy = \int_0^1 (x-x^2) dx = [\frac{x^2}{2} - \frac{x^3}{3}]_0^1 = \frac{1}{6}
  2. The joint density: f(x,y)={6,(x,y)∈D0,otherwisef(x,y) = \begin{cases} 6, & (x,y) \in D \\ 0, & \text{otherwise} \end{cases}
  3. E(X)=∬Dx⋅6 dx dy=6∫01x dx∫x2xdy=6∫01x(x−x2)dx=6∫01(x2−x3)dx=6[x33−x44]01=6(13−14)=6⋅112=12E(X) = \iint_D x \cdot 6 \, dx\,dy = 6\int_0^1 x \, dx \int_{x^2}^x dy = 6\int_0^1 x(x-x^2) dx = 6\int_0^1 (x^2-x^3) dx = 6[\frac{x^3}{3} - \frac{x^4}{4}]_0^1 = 6(\frac{1}{3} - \frac{1}{4}) = 6 \cdot \frac{1}{12} = \frac{1}{2}
  4. E(Y)=∬Dy⋅6 dx dy=6∫01dx∫x2xy dy=6∫01x2−x42dx=3∫01(x2−x4)dx=3[x33−x55]01=3(13−15)=3⋅215=25E(Y) = \iint_D y \cdot 6 \, dx\,dy = 6\int_0^1 dx \int_{x^2}^x y \, dy = 6\int_0^1 \frac{x^2-x^4}{2} dx = 3\int_0^1 (x^2-x^4) dx = 3[\frac{x^3}{3} - \frac{x^5}{5}]_0^1 = 3(\frac{1}{3} - \frac{1}{5}) = 3 \cdot \frac{2}{15} = \frac{2}{5}

Answer: E(X)=12E(X) = \frac{1}{2} and E(Y)=25E(Y) = \frac{2}{5}

Example 3: The Probability of a Normal Distribution

Suppose (X,Y)(X,Y) follows the two-dimensional normal distribution with joint density:

f(x,y)=12πe−x2+y22f(x,y) = \frac{1}{2\pi} e^{-\frac{x^2+y^2}{2}}

Find P{X2+Y2≤R2}P\{X^2 + Y^2 \leq R^2\}.

the two-dimensional normal distributionpolar coordinatescomputing a probabilitythe chi-square distribution

Approach:

The region is a disk, so use polar coordinates.

Steps:

  1. P{X2+Y2≤R2}=∬x2+y2≤R212πe−x2+y22dx dyP\{X^2+Y^2 \leq R^2\} = \iint_{x^2+y^2 \leq R^2} \frac{1}{2\pi} e^{-\frac{x^2+y^2}{2}} dx\,dy
  2. Polar coordinates: =12π∫02πdθ∫0Re−ρ22⋅ρ dρ=∫0Rρe−ρ22dρ=[−e−ρ22]0R=1−e−R22= \frac{1}{2\pi} \int_0^{2\pi} d\theta \int_0^R e^{-\frac{\rho^2}{2}} \cdot \rho \, d\rho = \int_0^R \rho e^{-\frac{\rho^2}{2}} d\rho = [-e^{-\frac{\rho^2}{2}}]_0^R = 1 - e^{-\frac{R^2}{2}}

Answer: 1−e−R221 - e^{-\frac{R^2}{2}}

(This is the distribution function of the chi-square distribution with 2 degrees of freedom.)


Exercises

Exercise 1

Suppose the joint density of (X,Y)(X,Y) is f(x,y)={e−(x+y),x>0, y>00,otherwisef(x,y) = \begin{cases} e^{-(x+y)}, & x>0, \, y>0 \\ 0, & \text{otherwise} \end{cases}. Decide whether XX and YY are independent, and find P{X<Y}P\{X < Y\}.

independencethe exponential distributioncomputing a probabilitythe joint density

Approach:

Find the marginal densities and decide whether there is independence; then compute the probability with a double integral.

Steps:

  1. The marginal densities: fX(x)=∫0+∞e−(x+y)dy=e−xf_X(x) = \int_0^{+\infty} e^{-(x+y)} dy = e^{-x} (for x>0x>0) fY(y)=∫0+∞e−(x+y)dx=e−yf_Y(y) = \int_0^{+\infty} e^{-(x+y)} dx = e^{-y} (for y>0y>0)
  2. f(x,y)=e−x⋅e−y=fX(x)fY(y)f(x,y) = e^{-x} \cdot e^{-y} = f_X(x) f_Y(y), so X,YX, Y are independent.
  3. P{X<Y}=∫0+∞dx∫x+∞e−(x+y)dy=∫0+∞e−x⋅e−xdx=∫0+∞e−2xdx=12P\{X < Y\} = \int_0^{+\infty} dx \int_x^{+\infty} e^{-(x+y)} dy = \int_0^{+\infty} e^{-x} \cdot e^{-x} dx = \int_0^{+\infty} e^{-2x} dx = \frac{1}{2}

Answer: X,YX, Y are independent (both follow an exponential distribution with parameter 1), and P{X<Y}=12P\{X<Y\} = \frac{1}{2}.

Exercise 2

Suppose (X,Y)(X,Y) is uniformly distributed over the disk x2+y2≤R2x^2+y^2 \leq R^2. Find the covariance Cov(X,Y)\text{Cov}(X,Y).

a uniform distributionthe covariancesymmetrya disk

Approach:

Use the symmetry: E(X)=E(Y)=0E(X)=E(Y)=0, and then compute E(XY)E(XY).

Steps:

  1. The joint density: f(x,y)=1πR2f(x,y) = \frac{1}{\pi R^2} (for x2+y2≤R2x^2+y^2 \leq R^2)
  2. By symmetry, E(X)=0, E(Y)=0E(X) = 0, \, E(Y) = 0
  3. E(XY)=∬x2+y2≤R2xy⋅1πR2dx dyE(XY) = \iint_{x^2+y^2 \leq R^2} xy \cdot \frac{1}{\pi R^2} dx\,dy
  4. The integrand xyxy is an odd function of xx (or of yy), and the region of integration is symmetric about the yy-axis (or the xx-axis), so the integral is 0.
  5. Cov(X,Y)=E(XY)−EX⋅EY=0−0=0\text{Cov}(X,Y) = E(XY) - EX \cdot EY = 0 - 0 = 0

Answer: Cov(X,Y)=0\text{Cov}(X,Y) = 0

(Note: a zero covariance does not imply independence; in this problem X,YX, Y are in fact not independent.)


Summary

Symbols Used in This Article

SymbolTypePronunciation/ExplanationMeaning in This Article
f(x,y)f(x,y)Joint probability densityjoint probability densityThe joint density function of a two-dimensional random variable
fX(x)f_X(x)Marginal probability densitymarginal densityThe marginal probability density of XX
fY(y)f_Y(y)Marginal probability densitymarginal densityThe marginal probability density of YY
fX∣Y(x∣y)f_{X\mid Y}(x\mid y)Conditional probability densityconditional densityThe density of XX given Y=yY=y
E(X)E(X)Mathematical expectationexpected valueThe expectation of the random variable XX
Cov(X,Y)\text{Cov}(X,Y)CovariancecovarianceThe covariance of XX and YY
ρXY\rho_{XY}Correlation coefficientcorrelation coefficientThe correlation coefficient of XX and YY
D(X)D(X)VariancevarianceThe variance of the random variable XX

Key Terms

English termPronunciationNote
joint probability density/dʒɔɪnt ˌprɒbəˈbɪləti ˈdɛnsəti/the density function of a two-dimensional random variable
marginal probability density/ˈmɑːrdʒɪnəl ˌprɒbəˈbɪləti ˈdɛnsəti/the probability density of a single variable
conditional probability density/kənˈdɪʃənəl ˌprɒbəˈbɪləti ˈdɛnsəti/the density of a conditional distribution
expected value / expectation/ɪkˈspɛktɪd ˈvæljuː/the average value of a random variable
covariance/koʊˈvɛriəns/the extent to which two variables vary together
correlation coefficient/ˌkɒrəˈleɪʃən ˌkoʊɪˈfɪʃənt/the normalised covariance
independence/ˌɪndɪˈpɛndəns/no association between the random variables
uniform distribution/ˈjuːnɪfɔːrm ˌdɪstrɪˈbjuːʃən/a distribution with constant probability density
bivariate normal distribution/baɪˈvɛriət ˈnɔːrməl ˌdɪstrɪˈbjuːʃən/the two-dimensional normal distribution
chi-square distribution/kaɪ skwɛər ˌdɪstrɪˈbjuːʃən/the distribution of the sum of squares of standard normal variables

Found an issue with this content?

Tell us what could be improved.