Robotic Force Control Via A Reinforcement Learning Approach

This thesis investigates the use of reinforcement learning (RL) for robotic forcecontrol using a planar elbow manipulator tasked with applying force to a rigidsurface. Inspired by simplified but effective simulation environments such as Grid-World, our approach leverages Q-learning to train the manipulator in a discreteaction–continuous state setting. The robot agent learns through interaction to establishend-effector contact without prior knowledge of the system’s dynamics. Theproposed control strategy is implemented in a simulated environment that capturesthe manipulator’s kinematics and the interaction forces with a two-layered wall. Unliketraditional force control methods, which often require accurate dynamic modelsand extensive offline tuning, our approach enables fast learning with fewer than10,000 training iterations. This allows for real-time or near-real-time applicationpotential in physical systems. Simulation results demonstrate that the Q-learningagent successfully converges to a stable and effective contact establishment policy.The study contributes to bridging foundational reinforcement learning algorithmsand practical robotic control problems, highlighting the feasibility of lightweight,model-free learning architectures for force-regulated interaction tasks.