Self-Improving Video-to-Action Models

[arXiv] Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson and Vincent Sitzmann published a paper on arXiv proposing a self-supervised method, Mirror, that learns a video-to-action mapping by interacting with the environment, without requiring prior action data or hand-designed rewards. The authors state that executing the visual plans produced by video generation models typically requires an "inverse dynamics" model trained on large-scale manually collected video-action data, which severely limits scalability. Mirror executes its best guess of the underlying actions according to the visual plan, then trains itself on the resulting video-action pairs, gradually improving itself. The paper reports that on Atari, OGBench and DeepMind Control, Mirror outperforms leading observation-learning methods when using learned video plans, and approaches expert level when using ideal plans; in Minecraft survival mode, it is the first method to obtain an iron pickaxe through self-improvement alone, without rewards or expert action labels.