DEEPSTEREO: LEARNING TO PREDICT NEW VIEWS FROM REAL WORLD IMAGERY

摘要

A system and method of deep learning using deep networks to predict new views from existing images may generate and improve models and representations from large-scale data. This system and method of deep learning may employ a deep architecture performing new view synthesis directly from pixels, trained from large numbers of posed image sets. A system employing this type of deep network may produce pixels of an unseen view based on pixels of neighboring views, lending itself to applications in graphics generation.

主权项

1. A method, comprising:
accessing a plurality of posed image sets from a database, the plurality of posed image sets respectively corresponding to a plurality of scenes, each of the plurality of posed image sets including a plurality of views of a corresponding scene of the plurality of scenes; generating a requested view of a scene based on views selected from the plurality of views of the scene included in posed image set corresponding to the scene in accordance with an automated view generating algorithm, wherein the requested view of the scene is not included in the plurality of views of the scene included in the corresponding posed image set; comparing the view of the scene generated by the automated view generating algorithm to a known view of the scene; and updating the view generating algorithm based on the comparison.