- Searching across a space of models
- Structure and inner working of processes of interest are given by models, or "parametric families of probability distributions"
- Once you get a model want to know how well it fits the data
- Usually two ways of measuring goodness of fit:
- Least-squares estimation (needs no distributional assumptions): Linear regression, sum of squares error, proportion variance accounted for, RMSE
- Maximum likelihood estimation (lots of distributional assumptions): Sufficiency - complete information about parameter of interest in MLE estimator Consistency - true parameter value that generated the data recovered asymptotically. Efficiency - lowest possible variance of parameter estimates. Parametrization invariance - same MLE solution independent of parametrization used