Results for "trial-and-error"

Existential risk from AI systems.

Regulating access to large-scale compute.

A mismatch between training and deployment data distributions that can degrade model performance.

A branch of ML using multi-layer neural networks to learn hierarchical representations, often excelling in vision, speech, and language.

How well a model performs on new data drawn from the same (or similar) distribution as training.

Designing input features to expose useful structure (e.g., ratios, lags, aggregations), often crucial outside deep learning.

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

When information from evaluation data improperly influences training, inflating reported performance.

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

A gradient method using random minibatches for efficient training on large datasets.

A parameterized function composed of interconnected units organized in layers with nonlinear activations.

Mechanism that computes context-aware mixtures of representations; scales well and captures long-range dependencies.

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Architecture that retrieves relevant documents (e.g., from a vector DB) and conditions generation on them to reduce hallucinations.

A datastore optimized for similarity search over embeddings, enabling semantic retrieval at scale.

Model-generated content that is fluent but unsupported by evidence or incorrect; mitigated by grounding and verification.

Constraining outputs to retrieved or provided sources, often with citation, to improve factual reliability.

Automated detection/prevention of disallowed outputs (toxicity, self-harm, illegal instruction, etc.).

Feature attribution method grounded in cooperative game theory for explaining predictions in tabular settings.

Controlled experiment comparing variants by random assignment to estimate causal effects of changes.

Training across many devices/silos without centralizing raw data; aggregates updates, not data.

How many requests or tokens can be processed per unit time; affects scalability and cost.

Techniques that fine-tune small additional components rather than all weights to reduce compute and storage.

Training a smaller “student” model to mimic a larger “teacher,” often improving efficiency while retaining performance.

Scales logits before sampling; higher increases randomness/diversity, lower increases determinism.

Converts logits to probabilities by exponentiation and normalization; common in classification and LMs.

Models that process or generate multiple modalities, enabling vision-language tasks, speech, video understanding, etc.

The shape of the loss function over parameter space.

Adjusting learning rate over training to improve convergence.

Tradeoffs between many layers vs many neurons per layer.