LoRA

Reducing numeric precision of weights/activations to speed inference and reduce memory with acceptable accuracy loss.

Removing weights or neurons to shrink models and improve efficiency; can be structured or unstructured.

Training a smaller “student” model to mimic a larger “teacher,” often improving efficiency while retaining pe...

Search algorithm for generation that keeps top-k partial sequences; can improve likelihood but reduce diversity.

Tracking where data came from and how it was transformed; key for debugging and compliance.

Time from request to response; critical for real-time inference and UX.

Hardware resources used for training/inference; constrained by memory bandwidth, FLOPs, and parallelism.

Techniques that fine-tune small additional components rather than all weights to reduce compute and storage.