Q(s, a) leftarrow Q(s, a) + alpha [r + gamma max{a'} Q(s', a') - Q(s, a)]
Check the final answer first, then review the worked steps.
Answer
Q-learning update rule
Step-by-step solution
- Identify the formula: This is the Q-learning update rule used in reinforcement learning to update the Q-value of a state-action pair.
- Understand the components: $Q(s, a)$ is the current value, $\alpha$ is the learning rate, $r$ is the reward, $\gamma$ is the discount factor, and $\max_{a'} Q(s', a')$ is the estimated optimal future value.
- Interpret the update: The term inside the brackets, $[r + \gamma \max_{a'} Q(s', a') - Q(s, a)]$, represents the temporal difference error, which is the difference between the target value and the current estimate.