Reconstructing, transferring and editing of 3D surfaces

Abstract

The authoring of realistic 3D faces and making a 3D character ready for animation are timeconsuming and tedious tasks. This thesis first addresses the problem of transferring the animation setup between 3D characters. We transfer skeletons and skinning weights between characters with distinct mesh topologies. Our pipeline takes as inputs a source character rig (mesh, hierarchy of joints, and skinning weights) and a target character mesh. We compute joint locations and orientations that embed the source skeleton in the target mesh, as well as skinning weights to bind the target geometry to the new skeleton. We first compute the geometric correspondence between source and target meshes. The resulting geometric correspondence is then used to formulate attribute transfer as an energy minimization and filtering problem. We demonstrate our approach on a variety of source and target bipedal characters, varying in mesh topology and morphology. Afundamental task of transferring animation is to establish an accurate geometric correspondence between the two characters. State-of-the-art mapping methods that exist today are globally good, but imperfections often exist and are often localized to a specific region. Consequently, we propose a novel approach to improve a given surface mapping through local refinement. The approach receives an established mapping between two surfaces and follows four phases: (i) creation of a sparse set of landmarks in mismatching regions; (ii) segmentation with a low-distortion region-growing process based on flattening the segmented parts; (iii) optimization of the deformation of segmented parts to align the landmarks in the planar parameterization domain; and (iv) aggregation of the mappings from segments to update the surface mapping. In addition, we propose a new approach to deform the mesh in order to meet constraints (in our case, the landmark alignment of phase (iii)). We incrementally adjust the cotangent weights for the constraints and apply the deformation in a fashion that guarantees that the deformed mesh will be free of flipped faces and will have low conformal distortion. Our new deformation approach, Iterative Least Squares Conformal Mapping (ILSCM), outperforms other low-distortion deformation methods. Lastly, we propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute editing workflow. Current face modeling methods using 3DMM suffer from a lack of local control. We thus create a 3DMM by combining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions. Our local PCA-based approach uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive, while allowing accurate reconstruction. The editing controls we provide to the user are intuitive as they are extracted from anthropometric measurements found in the literature. Out of a large set of possible anthropometric measurements, we filter those that have meaningful generative power given the face data set. We bind the measurements to the part-based 3DMM through mapping matrices derived from our data set of facial scans.

Date	2 Apr 2021
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Eric Paquette (Supervisor)