Efficient 3-D telestration for local and remote robotic proctoring

摘要

An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.

主权项

1. A medical system comprising:
an image capture device for capturing stereo images of a work site, each stereo image including a first two-dimensional image and a second two-dimensional image, wherein corresponding points in the first and second two-dimensional images depict a common point in the work site and exhibit a disparity between the corresponding points so as to indicate a depth of the common point relative to the image capture device; a stereo display displaying the first and second two-dimensional images so as to provide a three-dimensional viewing experience; and a processor that is programmed to: receive information of a first telestration indicative of user selected points of interest of the first two-dimensional image; selectively match the selected points of interest of the first two-dimensional image to corresponding points of the second two-dimensional image to determine matched points of interest of the second two-dimensional image by computing a plurality of image offsets wherein each of the plurality of image offsets is computed using a different image matching method than all others of the plurality of image offsets, and by using one of the computed plurality of image offsets which is selected according at least partially to one or more confidence scores of the plurality of image offsets, wherein the matched points of interest correspond to a second telestration mark; and cause the first and second telestration marks to be displayed respectively over the first and second two-dimensional images on the stereo display such that the first telestration mark and the second telestration mark appear as a single three-dimensional telestration mark on the stereo display.