Natural human to robot remote control

摘要

The subject disclosure is directed towards controlling a robot based upon sensing a user's natural and intuitive movements and expressions. User movements and/or facial expressions are captured by an image and depth camera, resulting in skeletal data and/or image data that is used to control a robot's operation, e.g., in a real time, remote (e.g., over the Internet) telepresence session. Robot components that may be controlled include robot “expressions” (e.g., audiovisual data output by the robot), robot head movements, robot mobility drive operations (e.g., to propel and/or turn the robot), and robot manipulator operations, e.g., an arm-like mechanism and/or hand-like mechanism.

主权项

1. In a computing environment, a system comprising:
a control program implemented in a remote device relative to a robotic device and configured to receive image data and depth data captured at a location remote from the robotic device, convert the image data and the depth data into skeletal data, process the skeletal data into one or more action commands that when processed by the robotic device operate one or more components of the robotic device, and transmit the one or more action commands to the robotic device over a connection to control the robotic device.