## Continuous Probability Distributions

We consider distributions that have a continuous range of values. Discrete probability distributions where defined by a probability mass function. Analogously continuous probability distributions are defined by a probability density function.

(This is a section in the notes here.)

## Discrete Probability Distributions

There are some probability distributions that occur frequently. This is because they either have a particularly natural or simple construction. Or they arise as the limit of some simpler distribution. Here we cover

• Bernoulli random variables
• Binomial distribution
• Geometric distribution
• Poisson distribution.

(This is a section in the notes here.)

## Counting Principles

(This is a section in the notes here.)

Counting in Probability. If each outcome is equally likely, i.e. $\mathbb P( \omega ) = p$ for all $\omega \in \Omega$, then since

(where $|\Omega|$ is the number of outcomes in the set $\Omega$ ) it must be that

## Exponential Families

The exponential family of distributions are a particularly tractable, yet broad, class of probability distributions. They are tractable because of a particularly nice [Fenchel] duality relationship between natural parameters and moment parameters. Moment parameters can be estimated by taking the empirical mean of sufficient statistics and the duality relationship can then recover an estimate of the distributions natural parameters.