REAL-TIME SYSTEM FOR MULTI-MODAL 3D GEOSPATIAL MAPPING, OBJECT RECOGNITION, SCENE ANNOTATION AND ANALYTICS

摘要

A multi-sensor, multi-modal data collection, analysis, recognition, and visualization platform can be embodied in a navigation capable vehicle. The platform provides an automated tool that can integrate multi-modal sensor data including two-dimensional image data, three-dimensional image data, and motion, location, or orientation data, and create a visual representation of the integrated sensor data, in a live operational environment. An illustrative platform architecture incorporates modular domain-specific business analytics “plug ins” to provide real-time annotation of the visual representation with domain-specific markups.

主权项

1. A mobile computing device, comprising:
one or more processors, and, in communication with the one or more processors: one or more image sensors configured to obtain two-dimensional image data and three-dimensional image data; one or more non-transitory machine accessible storage media comprising instructions to cause the mobile computing device to: temporally and spatially align the two-dimensional image data and three-dimensional image data; generate a map representation of a geo-spatial area of the real world surroundings of the mobile computing device based on the temporally and spatially aligned two-dimensional and three-dimensional image data; and recognize a plurality of visual features in the map representation, using one or more computer vision algorithms to:
recognize larger-scale objects;recognize smaller-scale objects by iteratively performing context-free object identification and contextual object identification; andrecognize a complex object comprising a plurality of the smaller-scale objects, using a classifier.