Lecture 14: Multivariate
Distributions
Probability Theory and Applications
Fall 2005
October 25
Multivariate Distributions
Distributions may have more than one R.V.
Example: S=size of house - real RV
P=price of house - real RV
A=Age of house - real RV
C= condition of house
Excellent, Very Good, Good, Poor
- discrete RV
Since variables are not-independent need a
multivariate distribution to describe them:
f(S,P,A,C)
Bivariate Random Variables
Given R.V. X and Y
Cases
1. X,Y both discrete
number of blue and red jelly beans
picked from jar
2. X,Y both continuous
height and weight
3. X discrete and Y continuous
date and stock price
Both Discrete
The joint distribution of (X,Y) is specified by
• The value set of (X,Y)
• The joint probability function
f(x,y)=P(X=x,Y=y)
Note:
• f(x,y)≥0 for any (x,y)
 f ( x, y)  1
x
y
Discrete Example
3H
2M
2D
Box contains jewels
H=high quality
M=medium quality
D=defective
You pick two jewels w/o replacement
X=# of H
 2
 
2 1

Y =#of M
P( X  0, Y  0) 

7
 
 2
21
Joint Probability Function
X\Y
0
1
2
0
1/21
4/21
1/21
6/21
1
6/21
6/21
0/21
12/21
2
3/21
0/21
0/21
3/21
10/21 10/21
1/21
Joint Probability Function
X\Y
0
1
2
0
1/21
4/21
1/21
1
6/21
6/21
0/21
2
3/21
0/21
0/21
Marginal Probability Functions
X\Y
0
1
2
f X ( x)
0
1/21
4/21
1/21
6/21
1
6/21
6/21
0/21
12/21
2
3/21
0/21
0/21
3/21
10/21 10/21
1/21
1
fY ( y )
Definitions
The marginal distribution of X is
f X ( x) 

f ( x, y )
yvalueset
Note this is exactly the same as pdf of X
The joint cumulative density function of X,Y
is
F ( x, y )  P ( X  x , Y  y )
Questions
P(You get one high quality and one medium
jewel)?
P(You pick at least one high quality jewel)?
Conditional Distributions
The conditional distribution of Y given X is
P(Y  y, X  x)
f ( y | x)  P(Y  y | X  x) 
P( X  x)
In our example:
P( x  1, y) f (1, y)
f ( y | x  1) 

P( X  1)
f x (1)
Conditional Probability Functions
X\Y
0
1
2
f X ( x)
0
1/21
4/21
1/21
6/21
1
6/21
6/21
0/21
12/21
2
3/21
0/21
0/21
3/21
fY ( y) 10/21 10/21
1/21
1
Y
f(y|X=1)
0
1
6 / 21
12 / 21
6 / 21
12 / 21
2
0 / 21
12 / 21
Conditional Probability Functions
X\Y
0
1
2
f X ( x)
0
1/21
4/21
1/21
6/21
1
6/21
6/21
0/21
12/21
2
3/21
0/21
0/21
3/21
fY ( y) 10/21 10/21
1/21
1
Find distribution of
X given Y=1
X
0
1
2
f(x|Y=1)
4/10
6/10
0
Question
Given that exactly one jewel picked is
medium quality, what is the probability
that the other is high quality?
6/10
Given that at least one jewel picked is
medium quality, what is the probability that
the other is high quality?
6/11
X,Y both Continuous
The joint pdf, f(x,y) defined over R2 has
properties:
 
• f(x,y)≥0
  f ( x, y)dxdy  1
To calculate probabilities, integrate joint pdf
over X,Y over the area
b d
P(a  X  b, c  Y  d )    f ( x, y )dydx
a c
Or more generally if we want P((x,y)A)
P( X , Y )  A)   f ( x, y)dA
A
X,Y both Continuous
More generally if we want P((x,y)A)
P( X , Y )  A)   f ( x, y)dA
A
The c.d.f.
F ( x, y )  P ( X  x , Y  y )
u
v

f (u, v)dvdu
 2 F ( x, y)
 f ( x, y)
xy
c.d . f

 
Marginals and Conditionals
The marginal pdf of X

f x ( x) 

f ( x, y )dy

The marginal pdf
of Y

f y ( y )   f ( x, y )dx

The conditional pdf of X given Y=y
f ( x, y)
f ( x | y) 
f y ( y)
Examples
The joint pdf of (x,y) is
f ( x, y )  cx(1  y )
Find c
1  c
1

2
0 0
1
c  1/ 2
1
x(1  y ) dydx  c  x (1  y ) dx |
 c  4 x dx  2c
0
for 0  X  1, 0  y  2
0
2
0
continued
2 x
f X ( x)  
0
2

1
f ( x, y )dy   x(1  y )dy  2 x
2
0
Find pdf of X
2
f X ( x)  
0
0  x 1
o.w.
Find pdf of Y
1
fY ( y )  
0
1
1
f ( x, y )dx   x(1  y )dx  1/ 4(1  y )
2
0
1/ 4(1  y)
fY ( y )  
0

0 y2
o.w.
continued
Find marginal of X given Y=1
f ( x, y  1) 1/ 2 x(1  1)
f ( x | y  1) 

 2x
fY (1)
1/ 4(1  1)
2 x
f X |Y ( x | y  1)  
0
0  x 1
o.w.
Note this is the same as marginal of X!
X and Y are independent!
f ( x, y ) 1/ 2 x(1  y )
f ( x | y) 

 2x
fY ( y )
1/ 4(1  y)
0  x 1
continued
Y
Find P(X>Y)
P( X  Y )  
?
?
P( X  Y )  
1

x
0 0

?
?
x
(1  y ) dydx
2
x
(1  y ) dydx
2
x
1 
y 
x  y   dx
0 2
2 0



2
1
1
1
x
1 x x 
1
2
x

dx



(1/ 3  1/ 8)  11/ 48



0
2
2
2 3 8 0 2
1
3
2
3
4
0
X
1
Mixed Continuous and Discrete
Let L a be R.V. that is 1 if candy corn
manufactured from Line 1 and 0 if line 0
Let X=weight of candy corn

  x  
1
The joint pdf is
0.25
e


( x  7.05)2
2
2


( x 10.1)2

1

f X , L ( x, l )  0.75
e 2(1.44)
2 1.2

0



L 1
  x  
L0
o.w.
What is the marginal distribution of X – the weight
of the candy corn?
Mixed Continuous and Discrete
The joint pdf is

1  ( x 7.05)
e 2
0.25
2


( x 10.1)2

1

f X , L ( x, l )  0.75
e 2(1.44)
2 1.2

0



2
  x  
L 1
  x  
L0
o.w.
Sum over L to find the marginal of X
f x ( x)  f ( x,1)  f ( x,0)
1
 0.25
e
2
( x 7.05)2

2
1
 0.75
e
2 1.2
( x 10.1)2

2(1.44)
  x  
Conditional Distribution
What is the marginal of L?
 
1
e
dx  0.25
L 1
  0.25
2



( x 10.1)2

1

f L ( x, l )    0.75
e 2(1.44) dx  0.75 L  0
2 1.2
 

.


L is Bernoulli R.V. p=0.25
( x  7.05)2

2
What is the conditional X given L?
 1  ( x 7.05)
2
e

 2

( x 10.1) 2

1

f (x | l)  
e 2(1.44)
 2 1.2




2
L 1
  x  
L0
  x  
.
If candy corn is from Line 1,
weight is normal with
mean 7.05 and s.d. = 1.
If candy corn is from Line 0,
weight is normal with
mean 10.1 and s.d. = 1.2.
Mixture Model
X is a mixture of two different normals
0.25 1/(sqrt(2 )) exp(-(x-7.05)2/2)+0.75 1/(sqrt(2  1.44)) exp(-(x-10.1)2/(2 1.44))
0.25
0.2
0.15
0.1
0.05
0
0
5
10
x
15
Example 5
Harry Potter plays flips a magical coin 10
times and records the number of heads.
The coin is magical because each day the
probability of getting heads changes.
Let Y, the probability of getting heads on a
given day, be uniform [0,1]
Let X be the number of heads of 10 gotten
on a given day with the magic coin.
What is the pdf of X?
Example 5 continued
Y is uniform [0,1] so
fY ( y )  1 0  y  1
X|Y is binomial n=10 p=Y
So f(X,Y)
10 
f ( X , Y )    y x (1  y )10 x
x
10  1 x
f X ( X )     y (1  y )10 x dy 
Note this is a Beta Integral!
 x 0
10  ( x  1)(10  x  1)
10!
x !(10  x)!_
 

11!
 x  ( x  1  10  x  1) (10  x)! x !

1
11
x  0,1,
,10
x  0,1,
,10, 0  y  1
X is discrete uniform
All values equally likely
Fact
You can compute the joint from a marginal
and a conditional.
f ( x, y)  f ( y | x) f x ( x)
Be careful how you compute the value sets!
Example 2 – Two Continuous
The joint pdf of X and Y is
f ( x, y )  cx(1  y )
1
0  x  y 1
Find marginal of X
1
1
f X ( x)   cx(1  y )dy  cx  y  y / 2 
x
x
Y
2
 cx 1  1/ 2  x  x 2 / 2 
O
X
1
2
3


c
x
/
2

x

x
/ 2
 
f X ( x)  
0


0  x 1
o.w.
Example 2
Still need c
1
1  c   x / 2  x 2  x 3 / 2 dx  c / 24  1  c  24
0
12 x(1  x)2
f X ( x)  
0

0  x 1
o.w.
You check:
12 y 2 (1  y)
fY ( y )  
0

0  y 1
o.w.
continued
P(Y≥2X)
Find P(Y<2X)
1
1
Y
Y
O
X
O
1
1
1

0
y
y/2
f ( x, y )dxdy  3 / 4

X
y/2
0 0
1
f ( x, y)dxdy  1/ 4
Conditional distribution
Find conditional pdf of Y and X=1/2
f (1/ 2, y )
f ( y | x  1/ 2) 
f x (1/ 2)
? y?
1/ 2  y  1
24 /(1/ 2)(1  y )
f ( y | x  1/ 2) 
12 / 2(1  1/ 2) 2
 8(1  y )
1
Y
O
X
1
8(1  y )
fY \ X 1/ 2 ( y | x  1/ 2)  
 0
1/ 2  y  1
o.w.
Conditional distribution
Find conditional pdf of Y and X=x 0<x<1
 2(1  y )

fY \ X 1/ 2 ( y | x)   1  x 2
 0
1
Y
O
X
1
x  y 1
o.w.
Independence
R.V. X and Y are independent if and only
any of the following hold
1. F(x,y)=FX(x)FY(y)
P(X≤x,Y≤y)= P(X≤x)P(Y≤y)
2. f(x,y)=fX(x)fY(y)
3. f(y|x)=fY(y)
Example 3
Given the joint pdf of X,Y
f ( x, y )  8 xy
0  x  y 1
Use the marginal of X and the conditional
pdf of Y given X=x to determine if X and Y
are independent?
Answer
Find marginal of X
1
Y
1
f x ( x)   8 xydy
O
x
 4 x(1  x 2 )
0  x 1
Find conditional of Y given X
8 xy
2y
f ( y | x) 

2
4 x(1  x ) 1  x 2
0  x  y 1
Answer continued
Are they independent?
y
f y ( y )   8 xydx  4 y 3
0  y 1
0
f x ( x) f y ( y)  4x(1  x2 )4 y3  8xy  f ( x, y)
No
Note
P(Y≤3/4|x=1/2) and P(Y≤3/4|x ≤1/2) are
very different things!
Let’s calculate each one
P(Y≤3/4|X=1/2)
The pdf of Y given X=1/2 is
2y
1  (1/ 2) 2
1/ 2  y  1
so
3/ 4
P(Y  3 / 4 | X  1/ 2) 

1/ 2
3/ 4
2y
f ( y | x  1/ 2)dy  
dy  5 /12
0.75
1/ 2
P(Y≤3/4|X ≤ 1/2)
The probability Y given X ≤ 1/2 is
P(Y  3/ 4 | X  1/ 2) 
P(Y  3/ 4, X  1/ 2)
P( X  1/ 2)
where
1/ 2
P( X  1/ 2) 

0
1/ 2
f x ( x)dx 

0
4 x(1  x 2 )dx  7 /16
P(Y≤3/4|X ≤ 1/2)
The probability P(Y≤3/4,X ≤ 1/2)
1
1/ 2 3/ 4
P(Y  3 / 4, X  1/ 2) 
  8 xydydx
0
1/ 2

x
 4x y
0
2 3/ 4
x
dx
O
1/ 2


4 x(9 /16  x 2 )dx  7 / 32
0
The probability
7 / 32
P(Y  3 / 4 | X  1/ 2) 
 1/ 2
7 /16
Example 4
Suppose X has the Gamma distribution with
parameters with K=2 and theta=1 and
f x ( x)  e  x x
0 x
the conditional distribution of Y given X.
(X>0) is
f ( y | x)  1/ x
Find P( X<4| Y=2)
0 y x
Example 4
We know f(x,y)=f(x|y)fx(x) so the joint is
1
f ( x, y )  e x
x
x
0 y x
The marginal of Y is

f y ( y)   e x dx  e y
0 y
y
Thus conditional of X given Y is
e x
f ( x | y)   y
e
y  x   for y  0
Example 4 continued
So
e x
f ( x | y  1/ 2)  1/ 2
e
1/ 2  x  
Thus
e x
P( x  4 | y  1/ 2)   1/ 2 dx  e 2
e
1/ 2
4
Exercise try: P(X>4|Y>2)
Example 5 – Two Discretes
You write a paper with an average rate of 10 errors
per paper. Assume the number of errors per
papers follows a Poisson distribution.
You roommate proofreads it for you, and he/she
has .8 percent of correcting each error.
What is the joint distributions of the number of
errors and the number of corrections?
What is the distribution of the number of errors
after you roommate reads the paper?
answer
Let X be the number of errors
Y be the number of errors after
correction
Clearly Y depends on X.
Given f X ( x) ~ Poisson(  10)
What is pdf of Y|X?
binomial(n=X,p=.2)
 x  y x y
f ( y | x)    .2 .8
 y
y  0,
,x
answer
Let X be the number of errors
Y be the number of errors after
correction
f X ,Y ( x, y)  f ( y | x) f x ( x)
 x  y x  y 10x e10
   .2 .8
x!
 y
Extra Credit: if you can figure out marginal of Y.
x  0, y  0,
,x