Imitation learning provides a simple recipe for learning complex behavior: instead of searching for bespoke exploratory policies or solving challenging reward design problems, one can collect expert demonstrations and apply supervised learning. While this paradigm has existed for many decades, it has seen an explosion of recent interest due to advances in robotics and applications to language models. Prior art has emphasized the role that the horizon of a task plays in governing the difficulty of learning a given policy and has cast doubt on the ability of supervised learning alone (referred to as Behavior Cloning in the context of imitation learning) to adequately learn in long-horizon tasks. I will discuss recent results showing that behavior cloning can achieve surprisingly strong guarantees and is optimal in many settings, both theoretically and empirically, as well as some examples of settings in which it can be improved upon. We will then move beyond the statistical question and examine the extent to which the presence of horizon fundamentally changes the optimization problem, as well as several mitigations, with applications to language model training.

https://upenn.zoom.us/j/97645937545

Meeting ID: 976 4593 7545