What is the relationship between poisson, gamma, and exponential distribution?

Solution 1:

Poisson and exponential distributions are very strongly related but they're fundamentally different because the Poisson is discrete (a count variable) and the exponential is continuous (a waiting time).

So how are they related?

If the time between a certain type of event is exponentially distributed with rate $\lambda$, then the number of events in a given time period of length $t$ follows a Poisson distribution with parameter $\lambda t$.

For example, if shooting stars appear in the sky at a rate of $\lambda$ per unit time, then the time you wait until you see your first shooting star is distributed exponentially with rate $\lambda$. If you watch the night sky for $t$ units of time, then you could see $0, 1, 2, ...$ shooting stars. The number of shooting stars that you count in this time is a $\text{Poisson}(\lambda t)$ random variable.

But what if you ask, how long must I wait before I see $n$ shooting stars?

The answer is a sum of independent exponentially distributed random variables, and it follows a $\text{gamma}(\lambda, n)$ distribution (also sometimes called an Erlang distribution, to distinguish it from the general gamma distribution where $n$ is allowed to be a non-integer).