Transforming random variables, joint and marginal distributions, and The Rule of the Lazy Statistician

## HW on transformation of random variables and joint density functions

I've been catching up on probability homework from chapter 2.

In this problem I had to prove this theorem:

Let $F$ be a cdf, and $U$ a random variable uniformly distributed on $[0, 1]$. Then $F^{-1}(U)$ is a random variable with cdf $F$.

After looking carefully at the definitions of inverse (or quantile) CDFs and reviewing the techniques for how to transform a random variable by applying a function to it, I finally (mostly) got it right before checking with the solution. I wasn't precise in thinking where the resulting random variable was defined, but got the main point. Was a good review of working carefully with the definitions of random variables and cumulative distribution functions.

Let $X$ have a CDF $F$. Find the CDF of $X^+ = \text{max}\{0,X\}$.

was more straight forward and again reviewed the technique of transforming a random variable.

These two problems both concerned joint density functions. In both cases, thinking clearly about how to integrate over 2d regions was the key.

## Marginal Distributions

I read about marginal distributions: you can integrate out a random variable from the joint distribution to get back to a distribution without that variable.

For joint density function $f_{X,Y}$ the marginal density $f_X(x) = \int f(x,y)dy$

In the discrete case, you sum over all values of the variable you are marginalizing.

$f_X(x) = P(X=x) = \sum_y P(X = x, Y = y) = \sum_y f(x,y)$

## Studying Expectation

I began reading ahead into chapter 3 and watching math monk probability videos on expectation. The formulas are pretty straight forward for both the discrete and continuous cases. Some tidbits that were new to me:

- A random variable may not have a well defined expectation
- A random variable may have a well defined infinite expectation

### Well defined

In order to determine whether a distribution is well defined, break it up into $ > 0$ and $< 0$ cases (negative and positive parts) and so long as one of them is finite, then the entire summation / integral is "well defined".

An example of a continuous random variable $X$ with an undefined expectation is The Cauchy distribution.

$ f(x) = \frac {1}{\pi(1 + x^2)}$

$ E(X) = \int_{-\infty}^{\infty} \frac {x}{\pi(1 + x^2)} $

It can be shown that both $\int_{-\infty}^{0} \frac {x}{\pi(1 + x^2)} $ and $\int_{0}^{\infty} \frac {x}{\pi(1 + x^2)} $ are infinite, making the sum of the two undefined. This is a bit hand wavy (see Wikipedia for a more rigorous description), but gives the intuition behind why this integral is undefined: you can't add negative infinity and positive infinity together and have a well defined value.

### Expectation rule aka The Rule of the Lazy Statistician

Another interesting property concerns how to compute the expectation of a function of a random variable.

What if we know $E(X)$ for some density function $f_x(x)$ and for some function $g(x)$ wish to know $E(g(x))$, but don't know $g_x(x)$?

The rule of the Lazy Statistician says we can plug $g(x)$ in for $x$ as follows:

$E(g(x)) = \sum_x g(x) f_X(x)$

and for the continuous case:

$E(g(x)) = \int_{-\infty}^{\infty} g(x) f_X(x) dx$

This seems pretty handy; we know the expected value of uniform distributions, normal distributions and a host of others, so if we would like to find the expected value of a random variable $Y$ with a PDF $f_Y(x)$ that can be re-written as a function of a random variable's PDF that we already know the expected value for, we can go that route without having to compute the integral $\int_{-\infty}^{\infty} x g_X(x) dx$