Chain-of-Thought

Architecture that retrieves relevant documents (e.g., from a vector DB) and conditions generation on them to reduce h...

Retrieval based on embedding similarity rather than keyword overlap, capturing paraphrases and related concepts.

Breaking documents into pieces for retrieval; chunk size/overlap strongly affect RAG quality.

Constraining outputs to retrieved or provided sources, often with citation, to improve factual reliability.

Training objective where the model predicts the next token given previous tokens (causal modeling).

Generates sequences one token at a time, conditioning on past tokens.

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Achieving task performance by providing a small number of examples inside the prompt without weight updates.