Deliberative Behavior Planner for UGV with Actively Articulated Suspension on Unstructured Terrain Using Q-Learning with Expert Model

Abstract

In this paper, we propose deliberative upper-level behavior planner as a part of the algorithm for UGV with actively articulated suspension to pass through the obstacle. We apply the Q-learning to this planning problem. To improve the convergence speed, we applied the expert model and action filter, and applied the conservative ε-greedy action selection method to shorten the behavior plans that is transferred to lower level behavior planner. Also we modified the centipede locomotion pattern and adapted to the suspensions’ movements for negotiating the obstacle. As a result, Q-learning with expert model could converge to the feasible behavior plans for various dimensions of the step up and down obstacles by simulation environment in about 10 seconds, which is acceptable speed for upper-level behavior planner. In addition, as a result of ANOVA test, conservative ε-greedy action selection method could shorten the behavior length significantly, but action filter did not work significantly.