Integration of causal inference in the DQN sampling process for classical control problems

Abstract

In this study, causal inference is integrated into deep reinforcement learning to enhance sampling in classical control environments. The problem we’re working on is "classical control," where an agent makes decisions to keep systems balanced. With the help of artificial intelligence and causal inference, we have developed a method that adjusts a deep Q-network’s experience memory by adjusting the priority of transitions. According to the agent’s actions, these priorities are based on the magnitude of causal differences. We have applied our methodology to a reference environment in reinforcement learning. In comparison with a deep Q-network based on conventional random sampling, the results indicate significant improvements in performance and learning efficiency. Our study shows that causal inference can be integrated into the sampling process so that experience transitions can be selected more intelligently, resulting in more effective learning for classical control problems. The study contributes to the convergence between artificial intelligence and causal inference, offering new perspectives for the application of reinforcement learning techniques in real-world applications where precise control is essential.