Surface segmentation from RGB and depth images

摘要

Surface segmentation from RGB and depth images is described. In one example, a computer receives an image of a scene. The image has pixels which each have an associated color value and an associated depth value representing a distance between from an image sensor to a surface in the scene. The computer uses the depth values to derive a set of three-dimensional planes present within the scene. A cost function is used to determine whether each pixel belongs to one of the planes, and the image elements are labeled accordingly. The cost function has terms dependent on the depth value of a pixel, and the color values of the pixels and at least one neighboring pixel. In various examples, the planes can be extended until they intersect to determine the extent of the scene, and pixels not belonging to a plane can be labeled as objects on the surfaces.

主权项

1. A computer-implemented method of image segmentation comprising:
receiving, at a processor, an image of a scene comprising a plurality of image elements, each image element having an associated color value and an associated depth value representing a distance between from an image sensor to a scene element; applying a weighted line scoring calculation to a plurality of straight lines extracted from the image to determine the principal directions for the image, the weighted line scoring calculation accounting for at least the surface normal of an image element, the direction of a straight line associated with the image element, and a plurality of predetermined weights; using the depth values to derive a set of three-dimensional planes present within the scene; and for each image element, determining whether the image element belongs to a plane from the set and labeling the image element accordingly, wherein the determining comprises evaluating a cost function over the set of planes having terms dependent on the depth value of the image element, the cost function having terms at least related to the number of planes, the set of derived planes, a unary term related to the 3D coordinates of the image element, and a unary term related to the normal of the image element, and the color value of the image element and at least one neighboring image element.