Bellman Equation

Expected cumulative reward from a state or state-action pair.

Optimizing policies directly via gradient ascent on expected reward.

Combines value estimation (critic) with policy learning (actor).

Learning from data generated by a different policy.

Learning only from current policy’s data.

Set of all actions available to the agent.

Strategy mapping states to actions.

Expected cumulative reward from a state or state-action pair.

Expected return of taking action in a state.