http://times.cs.uiuc.edu/course/410/note/mle.pdf

  • Searching across a space of models
  • Structure and inner working of processes of interest are given by models, or "parametric families of probability distributions"
  • Once you get a model want to know how well it fits the data
  • Usually two ways of measuring goodness of fit:
  1. Least-squares estimation (needs no distributional assumptions): Linear regression, sum of squares error, proportion variance accounted for, RMSE
  2. Maximum likelihood estimation (lots of distributional assumptions): Sufficiency - complete information about parameter of interest in MLE estimator Consistency - true parameter value that generated the data recovered asymptotically. Efficiency - lowest possible variance of parameter estimates. Parametrization invariance - same MLE solution independent of parametrization used