Data-Efficient Reinforcement Learning
The Dilemma: Reinforcement learning algorithms have been successful sometimes outperforming world-champions in different games. That said, they require tens-of-millions of agent-environmental interactions and months of cluster usage for successful behaviour.
The Potential Solution: To make reinforcement learning algorithms more data-efficient, I am mostly interested in two directions. The first, is transfer learning, while the second is model-based-type of algorithms. Transfer learning allows agents to re-use old knowledge acquired, while building models lead to surrogate environments to interact with. Both these directions hold the promise to make reinforcement learning much more data-efficient.
Title: An Information-Theoretic Optimality Principle for Deep Reinforcement Learning
Venue: Reinforcement Learning Workshop, NeurIps 2019
Authors: Felix Leibfried, Jordi Grau-Moya, Haitham Bou Ammar
Title: Model-Based Stabilisation of Deep Reinforcement Learning
Venue: arXiv e-Prints, 2018
Authors: Felix Leibfried, Rasul Tutunov, Peter Vrancx, Haitham Bou Ammar
Title: Learning High-Level Representation from Demonstrations
Venue: arXiv e-Prints, 2018
Authors: Garrett Anderson, Peter Vrancx, Haitham Bou Ammar
Title: Balancing Two-Player Stochastic Games with Soft Q-Learning
Venue: IJCAI, 2018
Authors: Jordi Grau-Moya, Felix Leibfried, Haitham Bou Ammar
Lifelong Machine Learning for Continual Knowledge
The Dilemma: Most state-of-the-art machine learning algorithms are tabula-rasa learning tasks in isolation, making them specific and not general.
The Potential Solution: To make machine learning algorithms generalise beyond their specific area, lifelong and continual learning is key. In this field I am interested in making machine learning algorithms acquire continual knowledge, which can then be combined in a novel way in new tasks for efficient learning.
Title: Scalable Lifelong Reinforcement Learning
Venue: Patter Recognition Journal, 2017
Authors: Yusen Zhan, Haitham Bou Ammar, Matthew E. Taylor
Title: Scalable Multitask Policy Gradients Reinforcement Learning
Venue: AAAI, 2017
Authors: Salam El-Bsat, Haitham Bou Ammar, Matthew E. Taylor
Venue: Conference on Decision and Control (CDC), 2017
Authors: Julia El-Zini, Rasul Tutunov, Haitham Bou Ammar, Ali Jadbabie
Title: Autonomous Cross-Domain Transfer in Lifelong Reinforcement Learning
Venue: IJCAI, 2015 (Best paper award nomination)
Authors: Haitham Bou Ammar, Eric Eaton, Jose Marcio Luna, Paul Ruvolo
Title: Safe Policy Search for Lifelong Reinforcement Learning with Sub-Linear Regret
Venue: ICML, 2015
Authors: Haitham Bou Ammar, Rasul Tutunov, Eric Eaton
Title: Online Multi-Task Learning for Policy Gradient Methods
Venue: ICML, 2014
Authors: Haitham Bou Ammar, Eric Eaton, Paul Ruvolo, Matthew E. Taylor
Distributed Optimisation Techniques
The Dilemma: Big data problems are so large to be stored on one machine. How are we, then, going to apply machine learning algorithms?
The Potential Solution: Rather than focusing on centralised solvers, I try to devise new distributed methods that require no central coordinator. We have shown that the techniques we developed are as good as centralised methods, yet much more efficient.