Incremental Feature Dependency Discovery

Abstract:

While well-designed state space representations are crucial to scaling existing reinforcement learning techniques to multi-agent domains, the benefit of gradual representational expansion is often overlooked. We introduce incremental Feature Dependency Discovery (iFDD), a computationally-inexpensive method for representation expansion that can be combined with any online, value-based reinforcement learning using binary features. The iFDD algorithm incrementally adds promising feature conjunctions in parts of the state space where feedback errors persist. In addition to convergence and computational complexity guarantees, when coupled with Sarsa, iFDD achieves much faster learning speed for standard Inverted Pendulum and BlocksWorld problems as well as for two real-world mission planning domains each with multiple unmanned aerial vehicles and hundreds of millions of state- action pairs.

Biography:

Alborz Geramifard is a PhD candidate in Aeronautics and Astronautics Department of MIT. He received his MSc in Computing Science from the University of Alberta at Edmonton in 2007. Before that, He obtained a BSc in Computer Engineering from the Sharif University of Technology at Tehran in 2003. His main research interest include machine learning with the focus on reinforcement learning, planning, and brain and cognitive sciences.