Improved Deep Reinforcement Learning for Efficient Motion Control of Autonomous Vehicle With Domain-Centralized Electronic and Electrical Architecture

With the rapid development of intelligent connected vehicles, the domain-based electronic and electrical (E/E) architecture is providing the potential upgrade for the autonomous vehicle. As a representative, the domain-centralized E/E architecture can be installed in the autonomous vehicle which performs its powerful software updates, cabling reduction, and functional integration. For the efficient and stable motion control of autonomous vehicles equipped with domain-centralized E/E architecture, this article proposes an improved deep reinforcement learning (RL) framework based on multihops loop delay and accelerated gradient optimization (LD-AGO-HQL). First, the domain-centralized E/E architecture and motion control problem of autonomous vehicle are modeled, respectively. Then, a multihops loop delay analysis (MH-LDA) is carried out for the E/E architecture to estimate the theoretical boundary value of heterogeneous topology loop delay. Subsequently, the deep RL algorithm using modified heuristic experience replay (MHER) is developed for the motion control of autonomous vehicle equipped with domain-centralized E/E architecture. In the implementation of deep RL system, the estimated loop delay value is integrated into the motion controller optimization, and the Nesterov accelerated gradient (NAG) is introduced and combined with the adaptive moment estimation (Adam) to improve the optimization effect. Finally, the real-world scenarios and virtual driving environment simulation are applied to evaluate the performance of the improved deep RL framework. The results show that the proposed motion control framework achieves better performance and guarantees the stability to the loop delay caused by domain-centralized E/E architecture.