Information Gain

A measure of randomness or uncertainty in a probability distribution.

Measures divergence between true and predicted probability distributions.

Measures how one probability distribution diverges from another.

Quantifies shared information between random variables.

A measure of a model class’s expressive capacity based on its ability to shatter datasets.

Measures a model’s ability to fit random noise; used to bound generalization error.

Systematic error introduced by simplifying assumptions in a learning algorithm.

Error due to sensitivity to fluctuations in the training dataset.