Vision-guided electromagnetic robotic system

摘要

Embodiments of the disclosure comprise a vision-based robotic manipulation system that identifies, singulates, and removes an individual surgical instrument from a cluttered environment. A computer-vision algorithm incorporated with the system is robust against optical challenges such as changing light conditions, specularities, and inter-reflections among various surgical instruments. The system estimates 2D pose (as opposed to a more challenging 3D pose) using a camera with normal room lighting to identify each object, including its specified data matrix barcode; and incorporates the use of a robotic arm with a compliant electromagnetic gripper to handle objects having large shape variance. The system can then relocate the objects as desired. The robot manipulator is utilized in hospital and research settings, manufacturing, and sterile environments.

主权项

1. A computer-implemented method of detecting and manipulating a plurality of instruments, the computer including a computer processor, and a camera with a barcode reading software module; the method comprising the steps of:
creating a template image of each of the instruments by way of the camera and storing the template images in memory; capturing an input image of the plurality of instruments; identifying each of the instruments by using the barcode reading software module that (i) reads an identification tag from each of the instruments and (ii) localizes reference points of the identification tag, each of the identification tags encoding a unique identifier; retrieving the template image from memory that corresponds to the identification tag; aligning one or more edges of an instrument in the input image with edges of the instrument in the template image to compute an initial affine transformation; computing an occupancy map that models one or more intersections between the instruments; and synthesizing occlusion relationships that correspond to one or more hypotheses, wherein the input image is compared against at least a first hypothesis and a second hypothesis;wherein the template image and the input image are two dimensional and utilized in combination to select a top-most instrument from a cluttered pile.