Human-in-the-Loop

Policies and practices for approving, monitoring, auditing, and documenting models in production.

Tendency to trust automated suggestions even when incorrect; mitigated by UI design, training, and checks.

Methods for breaking goals into steps; can be classical (A*, STRIPS) or LLM-driven with tool calls.

Mechanisms for retaining context across turns/sessions: scratchpads, vector memories, structured stores.

Coordinating tools, models, and steps (retrieval, calls, validation) to deliver reliable end-to-end behavior.

Hidden behavior activated by specific triggers, causing targeted mispredictions or undesired outputs.

Reconstructing a model or its capabilities via API queries or leaked artifacts.

Attacks that infer whether specific records were in training data, or reconstruct sensitive training examples.

Methods to protect model/data during inference (e.g., trusted execution environments) from operators/attackers.