A Framework for Empowering Reinforcement Learning Agents with Causal
  Analysis: Enhancing Automated Cryptocurrency Trading
Rasoul Amirzadeh Dhananjay Thiruvady Asef Nazari Mong Shan Ee
Despite advances in artificial intelligence-enhanced trading methods,
developing a profitable automated trading system remains challenging in the
rapidly evolving cryptocurrency market. This study aims to address these
challenges by developing a reinforcement learning-based automated trading
system for five popular altcoins~(cryptocurrencies other than Bitcoin): Binance
Coin, Ethereum, Litecoin, Ripple, and Tether. To this end, we present
CausalReinforceNet, a framework framed as a decision support system. Designed
as the foundational architecture of the trading system, the CausalReinforceNet
framework enhances the capabilities of the reinforcement learning agent through
causal analysis. Within this framework, we use Bayesian networks in the feature
engineering process to identify the most relevant features with causal
relationships that influence cryptocurrency price movements. Additionally, we
incorporate probabilistic price direction signals from dynamic Bayesian
networks to enhance our reinforcement learning agent's decision-making. Due to
the high volatility of the cryptocurrency market, we design our framework to
adopt a conservative approach that limits sell and buy position sizes to manage
risk. We develop two agents using the CausalReinforceNet framework, each based
on distinct reinforcement learning algorithms. The results indicate that our
framework substantially surpasses the Buy-and-Hold benchmark strategy in
profitability. Additionally, both agents generated notable returns on
investment for Binance Coin and Ethereum.