PROJECT TITLE :

Learning 3D Object Templates by Quantizing Geometry and Appearance Spaces

ABSTRACT:

While 3D object-focused form-based models are appealing compared with 2D viewer-centered look-primarily based models for their lower model complexities and potentially higher read generalizabilities, the educational and inference of 3D models has been much less studied within the recent literature because of two factors: i) the enormous complexities of 3D shapes in geometric house; and ii) the gap between 3D shapes and their appearances in pictures. This paper aims at tackling the 2 problems by studying an And-Or Tree (AoT) illustration that consists of 2 elements: i) a geometry-AoT quantizing the geometry space, i.e. the possible compositions of 3D volumetric elements and 2D surfaces within the volumes; and ii) an look-AoT quantizing the appearance area, i.e. the looks variations of those shapes in several views. In this AoT, an And-node decomposes an entity into constituent parts, and an Or-node represents alternative ways in which of decompositions. Therefore it will specific a combinatorial number of geometry and appearance configurations through tiny dictionaries of 3D shape primitives and 2D image primitives. In the quantized house, the problem of learning a 3D object template is transformed to a structure search problem which can be efficiently solved during a dynamic programming algorithm by maximizing the information gain. We tend to target learning 3D automotive templates from the AoT and collect a brand new car dataset that includes a lot of numerous views. The learned automobile templates integrate both the form-based model and the looks-primarily based model to combine the advantages of both. In experiments, we show 3 aspects: 1) the AoT is additional efficient than the frequently used octree technique in space representation; a pair of) the learned 3D automotive template matches the state-of-the art performances on car detection and pose estimation in a public multi-view car dataset; and 3) in our new dataset, the learned 3D template solves the joint task of simultaneous object detection, pose/view estimation, and half locali- ation. It can generalize over unseen views and performs higher than the version 5 of the DPM model in terms of object detection and semantic half localization.