3D human body mesh generation from 2D images using body silhouette, bone orientation, and joints triangulation

Abstract

In this dissertation, we address the problem of 3D human pose and shape estimation from multi-view images. Like similar methods, we make use of the Skinned Multi-Person Linear (SMPL) parametric body model, and try to regress the model parameters that best fit the shape and pose of the individual on the images. The main challenge lies in accurately inferring these parameters. To solve this problem, we first estimate 2D joints. Then, we use a linear algebraic triangulation to lift estimated 2D joints to 3D, resulting in a joint estimation with fewer errors. Next, we fit the 3D parametric body model to the 3D joints while imposing silhouette and bone orientation consistency between the 3D model and the detected individual in the images. We do so by minimizing a new set of objective functions through a two-step optimization process that provides a good initialization for the refinement of the shape and pose parameters. Finally, we demonstrate that the semantic position of joints in the body model and in the validation data sets do not exactly match. To account for this discrepancy we introduce, for each joint, a shift vector computed in the joint’s local space. Our fully automatic approach is evaluated on the widely used benchmarks Human3.6M and HumanEva, showing superior results with respect to state-of-the-art methods.

Date	21 Apr 2021
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Carlos Vázquez (Supervisor) & Éric Paquette (Co-supervisor)