Method and system for mutual interaction using space based augmentation

摘要

A method makes a first and a second devices support for interactions with respect to a 3D object. The method includes steps of: (a) allowing the first device to acquire information on physical 3D object and information on images of a user; (b) allowing the second device to receive the information relating to the physical 3D object and the information on images of the user of the first device, then display virtual 3D object corresponding to the physical 3D object and display 3D avatar of the user of the first device; (c) allowing the first device to transmit information on manipulation of the user of the first device regarding the physical 3D object and information on images of the user of the first device who is manipulating the physical 3D object and then allowing the second device to display the 3D avatar of the user of the first device.

主权项

1. A method for making a first and a second communication devices support for interactions with respect to a three-dimensional (3D) object, comprising steps of:
(a) allowing the first communication device to acquire information relating to a physical 3D object by taking shots of the physical 3D object in use of a camera module for augmentation and to acquire information on images of a user of the first communication device by taking shots of the user in use of a camera module for acquiring an image of a person; (b) allowing the second communication device to receive the information relating to the physical 3D object and the information on images of the user of the first, communication device from the first communication device, then display a virtual 3D object corresponding to the physical 3D object as a 3D image by using the received information relating to the physical 3D object and display a 3D avatar of the user of the first communication device by using the received information on the images of the user of the first communication device; (c) allowing the first communication device to transmit information on manipulation of the user of the first communication device regarding the physical 3D object and information on images of the user of the first communication device who is manipulating the physical 3D object, if being acquired, to the second communication device and then allowing the second communication device to display a state of the 3D avatar of the user of the first communication device who is manipulating the virtual 3D object by using a display unit and a projector of the second communication device; wherein, at the step of (a), the information relating to the physical 3D object includes information on color images and depth images and the first communication device acquires information on depth values of the physical 3D object by applying plane tracking for the depth images; and wherein the first communication device extracts plane information on a table on which the physical 3D object is placed by using a model based pose estimation and acquire the information on the depth values of the physical 3D object by subtracting depth values corresponding to the extracted plane information from depth values of the depth images.