Autonomous Soft Tissue Retraction Using Demonstration-Guided Reinforcement Learning

Abstract

In the context of surgery, robots can provide substantial assistance by performing small, repetitive tasks such as suturing, needle exchange, and tissue retraction, thereby enabling surgeons to concentrate on more complex aspects of the procedure. However, existing surgical task learning mainly pertains to rigid body interactions, whereas the advancement towards more sophisticated surgical robots necessitates the manipulation of soft bodies. Previous work focused on tissue phantoms for soft tissue task learning, which can be expensive and can be an entry barrier to research. Simulation environments present a safe and efficient way to learn surgical tasks before their application to actual tissue. In this study, we create a Robot Operating System (ROS)-compatible physics simulation environment with support for both rigid and soft body interactions within surgical tasks. Furthermore, we investigate the soft tissue interactions facilitated by the patient-side manipulator of the DaVinci surgical robot. Leveraging the pybullet physics engine, we simulate kinematics and establish anchor points to guide the robotic arm when manipulating soft tissue. Using demonstration-guided reinforcement learning (RL) algorithms, we investigate their performance in comparison to traditional reinforcement learning algorithms. Our in silico trials demonstrate a proof-of-concept for autonomous surgical soft tissue retraction. The results corroborate the feasibility of learning soft body manipulation through the application of reinforcement learning agents. This work lays the foundation for future research into the development and refinement of surgical robots capable of managing both rigid and soft tissue interactions.

Overview

In the present study, we discuss the complexities of soft tissue interactions with DaVinci robots and explore the use of a simulation environment to learn soft tissue manipulation tasks. To achieve this, we generate a simulation environment supporting soft and rigid body interactions and train agents to perform tissue retraction tasks. We formulate a rule-based policy to generate demonstration data to guide the training of reinforcement learning algorithms. In summary, our primary contributions to this work are threefold:

We establish a ROS-compatible physics simulation to replicate soft and rigid interactions for the DaVinci robot, particularly tailored to the tissue retraction task. We further explore the increased complexity of soft and rigid-body tasks compared to rigid-body interactions. In this context, we frame the task as a Markov Decision Process (MDP), elucidating the intricacies of such interactions.
We construct a rule-based policy for the generation of data utilized for task guidance, subsequently employing these data to train reinforcement learning agents with demonstration guidance.
We evaluate and compare the performance of demonstration-guided reinforcement learning agents with their traditional counterparts, observing performance dependence on the number of demonstrations available. We further perform an ablation study to see dependence on the number of demonstrations.

Performance Metrics

Task visualisation

BibTeX

  

        @misc{singh2023autonomous, title={Autonomous Soft Tissue Retraction Using Demonstration-Guided Reinforcement Learning}, author={Amritpal Singh and Wenqi Shi and May D Wang}, year={2023}, eprint={2309.00837}, archivePrefix={arXiv}, primaryClass={cs.LG} }