Basic Concepts
The normal distribution is characterized by two parameters: the mean μ and standard deviation σ (or variance σ2). Using the method of moments, we can estimate the distribution mean by the sample mean. We estimate the distribution standard deviation by the sample standard deviation.
Using the maximum likelihood estimation, once again we estimate the distribution mean by the sample mean. We estimate the standard deviation by the sample standard deviation but with division by n instead of n-1 (i.e. using VAR.P instead of VAR.S in Excel).
Log-likelihood Estimate
For a sample x1, …, xn, the log-likelihood is
when we use the sample variance s2 to estimate σ2 (method of moment approach). For the MLE approach, when the modified sample variance t2 is used, we get
Recall that s2 is an unbiased estimate of the variance, while t2 is a biased estimate (see Properties 3 and 7 of Estimators.
Example
Example 1: Estimate the mu and sigma parameters for a normal distribution that best fits the data in range A1:A18 of Figure 1. Use both the method of moments and MLE.
Figure 1 – Fitting a normal distribution
We display the MoM estimation on the upper right part of the figure and the MLE on the lower right part.
Examples Workbook
Click here to download the Excel workbook with the examples described on this webpage.
References
Wikipedia (2024) Maximum likelihood estimation
https://en.wikipedia.org/wiki/Maximum_likelihood_estimation
Carnegie Mellon University (2024) Maximum Likelihood Estimation
https://www.cs.cmu.edu/~10315/notes/10315_S24_Notes_MLE.pdf