Continuous Time Continuous Space Homeostatic Reinforcement Learning
  (CTCS-HRRL) : Towards Biological Self-Autonomous Agent
Hugo Laurencon Yesoda Bhargava Riddhi Zantye Charbel-Raphaël Ségerie Johann Lussange Veeky Baths Boris Gutkin
Homeostasis is a biological process by which living beings maintain their
internal balance. Previous research suggests that homeostasis is a learned
behaviour. Recently introduced Homeostatic Regulated Reinforcement Learning
(HRRL) framework attempts to explain this learned homeostatic behavior by
linking Drive Reduction Theory and Reinforcement Learning. This linkage has
been proven in the discrete time-space, but not in the continuous time-space.
In this work, we advance the HRRL framework to a continuous time-space
environment and validate the CTCS-HRRL (Continuous Time Continuous Space HRRL)
framework. We achieve this by designing a model that mimics the homeostatic
mechanisms in a real-world biological agent. This model uses the
Hamilton-Jacobian Bellman Equation, and function approximation based on neural
networks and Reinforcement Learning. Through a simulation-based experiment we
demonstrate the efficacy of this model and uncover the evidence linked to the
agent's ability to dynamically choose policies that favor homeostasis in a
continuously changing internal-state milieu. Results of our experiments
demonstrate that agent learns homeostatic behaviour in a CTCS environment,
making CTCS-HRRL a promising framework for modellng animal dynamics and
decision-making.