Moments and Moment Generating Functions
Reading Moments Out of a Moment Generating Function
A moment of a random variable is one number summarising one feature of its distribution. The nth raw moment is E[Xn]; the nth central moment is E[(X−μ)n], measured about the mean.
The first raw moment is the mean itself, the second central moment is the variance, and the third and fourth describe skewness and tail weight.
Computing these one at a time from a density or a mass function means a new integral or a new sum for every order.
One function that stores every moment
The moment generating function of X is
defined for those t where the expectation is finite. Expanding etX as a power series and taking expectations term by term gives
so the moments are stored as the Taylor coefficients. There are two equivalent ways to get one out. Differentiate n times and set t=0, which gives MX(n)(0)=E[Xn]; or, if you already have the series, read off the coefficient of tn and multiply it by n!. The variance then follows from the first two raw moments, Var(X)=MX′′(0)−[MX′(0)]2.
How the Taylor expansion of a moment generating function about the origin stores the raw moments of X, and the two equivalent ways of reading one out.
Worked example
Let X be 1 if a randomly chosen bus is late and 0 otherwise, with P(X=1)=0.4. Then MX(t)=0.6e0⋅t+0.4e1⋅t=0.6+0.4et. Differentiating, MX′(t)=0.4et, so MX′(0)=0.4=E[X]. Differentiating again gives MX′′(t)=0.4et and MX′′(0)=0.4=E[X2], which is no surprise because X2=X for a zero-one variable. The variance is 0.4−(0.4)2=0.24. Every moment came from one function and some differentiation, with no sums to redo.
Existence and validity checks
Two quick tests catch most errors. First, MX(0)=E[e0]=1 for every random variable, so a candidate function with M(0)=1 is not an MGF.
Second, the definition only helps if the expectation is finite on an open interval containing 0; for the exponential family of waiting times, for instance, MX(t) exists only for t below the rate, and outside that range the integral diverges.
Some distributions, such as the Cauchy, have no MGF at all, which is why probabilists fall back on the characteristic function E[eitX]. The characteristic function always exists and still characterizes the distribution.
When the relevant moments exist, its derivatives at zero can also recover them: if E∣X∣n is finite, then φ(n)(0)=inE[Xn]. Unlike an MGF, however, its existence alone does not guarantee that any moments exist; the Cauchy characteristic function e−∣t∣ is not even differentiable at zero.
Moment generating function of a bus waiting time
The figure shows how the Taylor expansion of MX(t) about t=0 stores the raw moments of X. A random variable X has a moment generating function whose expansion about t=0 begins
What is E[X2]?
How the Taylor expansion of a moment generating function about the origin stores the raw moments of X, and the two equivalent ways of reading one out.
Moment generating function of a bus waiting time
The graph shows the moment generating function of a bus waiting time X, in minutes, which is modelled as exponential with an unknown rate λ, so that MX(t)=λ−tλ for t<λ. The plotted curve passes through the point (2,2). What is E[X]?
How the Taylor expansion of a moment generating function about the origin stores the raw moments of X, and the two equivalent ways of reading one out.
Moment generating function of a bus waiting time