Our system FunGrasp can grasp diverse objects ranging from a large heavy hammer to thin light scissors.
Functional grasping is essential for humans to perform specific tasks, such as grasping scissors by the finger holes to cut materials or by the blade to safely hand them over. Enabling dexterous robot hands with functional grasping capabilities is crucial for their deployment to accomplish diverse real-world tasks. Recent research in dexterous grasping, however, often focuses on power grasps while overlooking task- and object-specific functional grasping poses. In this paper, we introduce FunGrasp, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects. Given a single RGBD image of functional human grasping, our system estimates the hand pose and transfers it to different robotic hands via a human-to-robot (H2R) grasp retargeting module. Guided by the retargeted grasping poses, a policy is trained through reinforcement learning in simulation for dynamic grasping control. To achieve robust sim-to-real transfer, we employ several techniques including privileged learning, system identification, domain randomization, and gravity compensation. In our experiments, we demonstrate that our system enables diverse functional grasping of unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands. The significance of the components is validated through comprehensive ablation studies.
Our system contains three modules: A) H2R Grasp Retargeting, which retargets the functional grasp poses from the human hand to diverse robot hands. B) Dynamic Grasp Control, which controls the robot hands to achieve dynamic grasping motions with a policy trained by RL. C) Sim-to-Real Transfer, which applies techniques including privileged learning and system identification for robust hardware deployment.
FunGrasp shows adaptive behavior to keep stable grasping.
Evaluation on Allegro hand and Inspire hand
FunGrasp supports real-time real-to-sim visualization in physics simulation.
@article{huang2024fungrasp,
title={{FunGrasp}: Functional Grasping for Diverse Dexterous Hands},
author={Huang, linyi and Zhang, Hui and Wu, Zijian and Christen, Sammy and Song, Jie},
journal={arXiv preprint arXiv:2411.16755},
year={2024}
}