DSE: A Denoising State Estimator for RL-Based Bipedal Robot Locomotion

Recent advancements in legged robot locomotion and reinforcement learning have demonstrated significant potential for the development of bipedal robot. But the state estimation accuracy and bipedal robot locomotion robustness of Reinforcement Learning based (RL-based) controller is significantly influenced by IMU's measurement noise. High-precision IMUs can obtain accurate information, but manufacturing cost is high, while robots equipped with low-price IMUs may face large noise and bias in-consistence. In this letter, we propose a novel denoising autoencoder-based state estimator (DSE) to address sensor noise cancellation and state estimation problem in RL-based bipedal robot locomotion control. The DSE architecture learns a compact representation of robots' system dynamics behind those low-price IMU's noisy data and provides noise reduced measurements and accurate state estimation for learning-based controller. We demonstrate the effectiveness of the DSE architecture in reducing noise and enhancing the robustness of both state estimation and locomotion control in various indoor and outdoor experiments. The results highlight the potential of DSE framework in facing noise distribution difference between simulation and reality.