Method for the real-time-capable, computer-assisted analysis of an image sequence containing a variable pose

摘要

The invention relates to a method for the real-time-capable, computer-assisted analysis of an image sequence of an object consisting of elements that can be moved relative to each other and are interconnected, said sequence containing a variable pose, wherein the individual images of the image sequence are recorded by way of a time-of-flight (TOF) camera such that said images can be processed by a computer, and contain brightness and distance data as functions of the pixel coordinates of the camera for each image of the sequence, comprising the following steps: a. Capturing the pixels of an individual image forming the object, b. calculating a three-dimensional (3D) point cloud in a virtual space, said point cloud representing the surface of the object that is visible to the camera, by a computational projection of object-depicting pixels in such a space, while taking captured distance data to the object into consideration, c. fitting a model of the object consisting of nodes and edges into the computer-generated 3D point cloud for the individual images, wherein the nodes represent a selection of elements of the object and the edges represent the connections of said elements amount each other, d. iteratively updating all node positions by applying a learning rule for training a self-organizing map having a previously defined number of randomly selected dots of the point cloud, e. repeating steps a. to d. for each subsequent individual image of the sequence, wherein for the fitting in step c. the result of step e. of the preceding image is used in each case, and f. determining the varying pose from the positions of predetermined nodes of the model which have been captured in at least representative images of the image sequence.

主权项

1. A method for real-time-capable computer-assisted analysis of an image sequence containing a variable pose of an object composed of interconnected elements movable in relation to each other, wherein the frames of the image sequence have been recorded by a distance measuring camera and are processed by a computer and the frames have brightness and distance data as functions of pixel coordinates of the camera for each frame of the sequence; the method comprising:
(a) the computer detecting the pixels of a frame which map the object; (b) the computer calculating a three-dimensional (3D) aggregate of points within a virtual space, which represents that surface of the object which is visible for the camera, by calculated projection of object-mapping pixels into such a space, taking into account acquired data of a distance from the object; (c) the computer fitting a model of the object, which consists of nodes and edges, into the computer-generated 3D aggregate of points for the frame, the nodes representing a selection of elements of the object and the edges representing interconnections of these elements; (d) the computer iteratively updating all node positions by using a learning rule for training a self-organizing map with a predetermined number of randomly sampled points of the aggregate of points; (e) repeating at least 25 times per second the steps (a) through (d) for each subsequent frame of the sequence, the result of step (d) of the preceding frame being used respectively for the fitting process in step (c); (f) the computer determining the changing pose from positions of predetermined nodes of the model, which have been detected in at least representative frames of the image sequence; (g) wherein the node positions of the model are updated in iteration steps, one point x being randomly sampled for the predetermined number of points of the 3D aggregate of points in each iteration step and all nodes being shifted towards x; and (h) the degree of the shift being largest for the node which had the smallest distance from x prior to the iteration step wherein the number of the randomly sampled points x or that of the iteration steps is about 10% of the total number of the points within the aggregate of points.