Results for "training loss"

Pixel-wise classification of image regions.

Loss of old knowledge when learning new tasks.

Learning policies from expert demonstrations.

Methods that learn training procedures or initializations so models can adapt quickly to new tasks with little data.

Techniques that discourage overly complex solutions to improve generalization (reduce overfitting).

How well a model performs on new data drawn from the same (or similar) distribution as training.

Controls the size of parameter updates; too high diverges, too low trains slowly or gets stuck.

Achieving task performance by providing a small number of examples inside the prompt without weight updates.

Gradients shrink through layers, slowing learning in early layers; mitigated by ReLU, residuals, normalization.

Fine-tuning on (prompt, response) pairs to align a model with instruction-following behaviors.

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.

Reducing numeric precision of weights/activations to speed inference and reduce memory with acceptable accuracy loss.

Flat high-dimensional regions slowing training.

Configuration choices not learned directly (or not typically learned) that govern training or architecture.

Number of samples per gradient update; impacts compute efficiency, generalization, and stability.

Gradually increasing learning rate at training start to avoid divergence.

Recovering training data from gradients.

Inferring sensitive features of training data.

Model behaves well during training but not deployment.

Differences between training and inference conditions.

Training one model on multiple tasks simultaneously to improve generalization through shared structure.

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Hidden behavior activated by specific triggers, causing targeted mispredictions or undesired outputs.

Autoencoder using probabilistic latent variables and KL regularization.

Generator produces limited variety of outputs.

Assigning category labels to images.

Learning action mapping directly from demonstrations.

Training with a small labeled dataset plus a larger unlabeled dataset, leveraging assumptions like smoothness/cluster structure.

A parameterized mapping from inputs to outputs; includes architecture + learned parameters.

When a model cannot capture underlying structure, performing poorly on both training and test data.