Abstract
The increasing deployment of robots in unstructured and human-centric environments demands sophisticated control strategies that can handle physical interaction safely and effectively. Impedance control is a fundamental approach for managing such interactions, yet its efficacy is often limited by the use of fixed, manually-tuned parameters that are suboptimal across different phases of a contact-rich task. This paper addresses the challenge of optimizing impedance control by proposing a novel framework based on Deep Reinforcement Learning (DRL). We formulate the variable impedance control problem as a continuous control task where a DRL agent learns to dynamically modulate the stiffness parameters of a manipulator's end-effector online. The agent's goal is to successfully complete a compliant manipulation task while minimizing interaction forces and control effort. We employ the Deep Deterministic Policy Gradient (DDPG) algorithm to train a policy that maps robot states and force/torque sensor readings to optimal impedance parameters. The proposed method is evaluated in a simulated peg-in-hole assembly task, a benchmark for contact-rich manipulation. Results demonstrate that the DRL-based variable impedance controller significantly outperforms conventional fixed-gain low- and high-stiffness controllers in terms of success rate, completion time, and peak interaction force. The learned policies exhibit intelligent, phase-dependent behaviors, adapting stiffness in real-time to navigate from free space to contact and insertion. This work establishes the viability of DRL as a powerful, model-free method for automating the synthesis of high-performing adaptive controllers for complex robotic interaction tasks.