Interactive Shaping of Granular Media Using Reinforcement Learning

Autonomous manipulation of granular media, such as sand, is crucial for applications in construction, excavation, and additive manufacturing. However, shaping granular materials presents unique challenges due to their high-dimensional configuration space and complex dynamics. Traditional rule-based approaches struggle with these complexities, requiring extensive engineering efforts. Reinforcement learning (RL) offers a promising alternative by enabling agents to learn adaptive manipulation strategies through trial and error. Although RL has been successfully applied to rigid and deformable object manipulation, its application to granular media has received little attention. Thus, it has remained an open research question how to define the compact observations for the large configuration space and design an effective reward function. In this work, we present an RL framework that enables a robotic arm with a cubic end-effector to shape granular media into desired structures, outperforming two baseline approaches. Our results demonstrate the effectiveness of the proposed reward formulation for the training of visual policies that manipulate granular media including their real-world deployment.