Man-machine Interaction System of Manipulator Based on Kinect V2 and Unity3D
LI Ziliang
LI Qingdang
WANG Xiaobo
HE Xuetai
Abstract：Aiming at the problems of unintuitive and complex operation in the human-computer interaction,this paper devel-opes a human-computer interaction system based on Kinect V2 and Unity3D by using the hand tracking technology.Users can con-trol the movement of the robot arm by waving their hands directly,and the interaction mode is more simple,intuitive and flexible.This system uses Kinect V2 to obtain the trajectory of hand movement to realize hand tracking,and develops the interactive system on the Unity3D platform.FABRIK algorithm and Bézier curve are used to solve the inverse kinematics and record the motion path,respectively.The required script function is written in C#,and two interactive modes are developed,which are called real-time con-trol mode and teaching mode,these two modes can be selected through the UI panel.Finally,the two modes are tested respectively,and the experimental results prove the feasibility of the system.
Keywords：human-computer interactionhand trackingKinect V2Unity3D
Publication Date：2024-03-20
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：6( 735-739,745 )