Causal Reinforcement Learning

Currently I am exploring a promising virgin field: Causal Reinforcement Learning (CausalRL). What has been inspiring me is the philosophy behind the integration of causal inference and reinforcement learning, that is, when looking back at the history of science, human beings always progress in a similar manner to that of CausalRL:

Humans summarize rules or experience from their interaction with nature and then exploit this to improve their adaptation in the next exploration. What CausalRL does is exactly to mimic human behaviors, i.e., learning causal relations from an agent that communicates with the environment and then optimizing its policy based on the learned causal structures.

The reason that I highlight this analogy is to emphasize the importance of CausalRL. Personally speaking, CausalRL will, without doubt, become an indispensable part of Artificial General Intelligence (AGI), which has great potential applications not only in healthcare and medicine but also in all other RL scenarios. Compared to RL, CausalRL has one obvious advantage inherited from causal inference: data efficiency.

Please refer to CausalRL for more details.


Deconfounding Reinforcement Learning in Observational Settings
Chaochao Lu, Bernhard Schölkopf, José Miguel Hernández-Lobato.
arXiv preprint arXiv:1812.10576 
[Paper] [Code]