Object detection and tracking for providing a virtual device experience

摘要

The technology disclosed can provide capabilities such as using vibrational sensors and/or other types of sensors coupled to a motion-capture system to monitor contact with a surface that a user can touch. A virtual device can be projected onto at least a portion of the surface. Location information of a user contact with the surface is determined based at least in part upon vibrations produced by the contact. Control information is communicated to a system based in part on a combination of the location on the surface portion of the virtual device and the detected location information of the user contact. The virtual device experience can be augmented in some implementations by the addition of haptic, audio and/or visual projectors.

主权项

1. A method of operating a motion-capture system, the method including:
monitoring at least three vibration sensors coupled to a motion-capture system; and in response to detection of vibrations generated by contact of an object with a surface of a first medium, tracking a position of the object by:
detecting a first vibration through the first medium by a first vibration sensor;detecting a second vibration through a second medium by at least a second and a third vibration sensor, wherein the first medium is different from the second medium;measuring a time difference of arrival (TDOA) of the second vibration at the second and third vibration sensors with respect to the TDOA of the first vibration at the first vibration sensor and mapping surface locations that satisfy the TDOA, whereby the TDOA of the second vibration at the second and third vibration sensors differs with respect to the TDOA of the first vibration at the first vibration sensor because the second vibration travels through the second medium at a different velocity that the first vibration travels through the first medium;calculating a distance from the object to the second and third vibration sensors based on the respective TDOAs; anddetermining location information for the object relative to the vibration sensors based on the mapped TDOA surface locations.