Non-Convex Optimization

A point where gradient is zero but is neither a max nor min; common in deep nets.

Variability introduced by minibatch sampling during SGD.

The shape of the loss function over parameter space.

A narrow minimum often associated with poorer generalization.

Estimating parameters by maximizing likelihood of observed data.

Updating beliefs about parameters using observed evidence and prior distributions.

Bayesian parameter estimation using the mode of the posterior distribution.

Optimization problems where any local minimum is global.