DAgger, short for Dataset Aggregation, is an 🐵 Imitation Learning method that enriches the expert dataset with corrections to address 🌀 Behavioral Cloning > Distributional Shift. Specifically, we loop the following:
- Train
from human data , following 🌀 Behavioral Cloning. - Run
to get dataset of states. - Ask an expert to label
with correct actions. - Aggregate
, and repeat.
By incorporating the policy's empirical states into our dataset, over many iterations, we'll have the data's distribution of observations converge to the policy's distribution,
thus allowing our model to learn correct responses to the states it encounters. Theoretically, this mitigates the distribution shift problem by essentially training directly on the policy's distribution.