Objective Function

A function measuring prediction error (and sometimes calibration), guiding gradient-based optimization.

Minimizing average loss on training data; can overfit when data is limited or biased.

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

Average of squared residuals; common regression objective.

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

Configuration choices not learned directly (or not typically learned) that govern training or architecture.