Visual servo algorithm of robot arm simulation for dynamic tracking and grasping application

Rishi Arran Suppramaniam, and Mohd Hairi Mohd Zaman, and Mohd Faisal Ibrahim, and Seri Mastura Mustaza, and Asraf Mohamed Moubark, (2022) Visual servo algorithm of robot arm simulation for dynamic tracking and grasping application. Jurnal Kejuruteraan, 34 (4). pp. 729-739. ISSN 0128-0198


Official URL:


Health pandemics such as Covid-19 have drastically shifted the world economics and boosted the development of automation technologies in the industries for continuous operation without human intervention. This paper elaborates on an approach to dynamically track and grasp moving objects using a robot arm. The robot arm has an eye-in-hand (EIH) configuration, where a camera is installed on the robot arm’s end effector. The working principle of the robot arm in this paper is mainly dependent on the recognition of augmented reality markers, i.e., Aruco markers, placed on the dynamically moving target object with continuous tracking. Then, the proposed system updates the predicted location for the markers using the Kalman filter for performing grasping. The proposed approach identifies the Aruco marker on the target object and estimates the object’s location using previous states, and performs grasping at the exact predicted location. When extracted information is updated, the vision system also implements a feedback control system for stability and reliability. The proposed approach is tested using simulation of the dynamic moving object with different speeds and directions. The robot arm with the Kalman filter can track and grasp the dynamic object at a speed of 0.2m/s with a 100% success rate while obtaining an 80% success rate at a speed of 0.3m/s. In conclusion, the moving object’s speed is directly proportional to the grasping time until it reaches the threshold speed for the camera in identifying the Aruco markers. Future works are required to improve the dynamic visual servo algorithm with motion planning when obstacles are present in the path of robot grasping.

Item Type:Article
Keywords:Eye-in-hand; Aruco markers; Kalman filter; Feedback controller; Velocity threshold
Journal:Jurnal Kejuruteraan
ID Code:20340
Deposited By: ms aida -
Deposited On:26 Oct 2022 07:39
Last Modified:02 Nov 2022 01:20

Repository Staff Only: item control page