Abstract

In the past few years, studying in the field of Augmented Reality (AR) has been expanded from technical aspect such as tracking system, authoring tools and etc. to applications ranging from the fields of education, entertainment, medicine to manufacturing. In manufacturing, which relies on assembly process, AR is used for assisting staffs in the field of maintenance and assembly. Usually, it has been used as a guidance system, for example using graphical instructions for advising the users with the steps in performing the maintenance or assembly operation. In assembly training, especially for small, expensive or harmful devices, interactive technique using real hands may be suitable than the guiding technique. Using tracking algorithm to track both hands in real time, interaction can occurs by the execution of grasp and release gestures. Bare hand tracking technique, which uses gesture recognition to enable interaction with augmented objects are also possible. In this paper, we attempted to use marker based AR technique to assemble 3D virtual objects using natural hand interaction. By applying the markers to fit on fingertip and assigned the corresponding virtual 3D finger that have physical properties such as surface, volume, density, friction and collision detection properties to them, interaction between fingers and objects could be executed. This setup was designed on a PC based system but could be ported to iOS or Android, so that it would work on tablet or mobile phones as well. Unity 3D game engine was used with Vuforia AR platform. In order to grab and move the virtual object by hand, the shape of the virtual finger (Vulforia's target) has been investigated. Appropriate friction coefficient were applied to both virtual fingers and the object and then at least two virtual fingers were force to press on the 3D virtual object in opposite directions so that frictional force is more than gravitational force. To test this method, virtual model of LEGO's mini-figures which composed of five pieces, was used and the assembly could be done in just a short time. Comparing with other popular technique such as "gestures recognition", we have found that our technique could provide more efficient result in term of cost and natural feeling. © 2014 Springer International Publishing.