Fully automatic CNN-Based personalized 3D femur reconstruction from EOS 2D Bi-planar radiographs

Abstract

Clinical 3D geometric measurements of the lower limb bones such as the femur, in standing position, are crucial in orthopedic pre-operative planning and patient follow-up. In clinical routine, the personalized 3D model reconstruction of the femur is a useful tool for physicians to analyze a complex 3D shape deformation. They use a 3D reconstructed femur to quantify clinical 3D geometric measurements such as size, curvatures, orientations, femoral-tibia rotation, and femoral torsion. 2D bi-planar radiographs-based 3D bone model reconstruction provides an efficient alternative to Computerized Tomography (CT) for orthopedic surgical planning and patient follow-up. CT-scan-based 3D bone model reconstruction methods suffer from high radiation dose, acquisition costs, and operation with patient in reclining position. In 2D bi-planar radiographs-based 3D model reconstruction, 3D/2D registration is an essential task to establish a geometric relationship between a known prior 3D model and a patient’s 2D bi-planar radiograph. This registration process includes the 3D pose and the 3D shape estimation of bone structures from only two 2D projections, is highly complex due to information loss during 2D projection of 3D bone and the need to solve an inverse problem using 2D projected sparse data. Semi-automatic methods, such as the one employed by the EOS® 3D model reconstruction system, require the manual intervention of an operator for the pose initialization and the shape and scale adjustment of the 3D model to the images. These manual interventions impact the accuracy, time-efficiency, and reproducibility of the approaches. In this thesis, we develop a fully automatic EOS® 2D bi-planar radiographsbased personalized 3D femur reconstruction via deep learning approaches. The developed fully automatic personalized 3D femur reconstruction workflow crosscuts two main stages. Firstly, 3D bone pose and isotropic scale estimation, and secondly, 3D/2D nonrigid registration (3D shape deformation). In the first stage, an automatic coarse-to-fine 3D/2D similarity registration method is proposed to automatically register a generic 3D model of the femur into EOS® 2D bi-planar radiographs acquired with two different fields of view, full body and whole lower limbs, and patients’ orientations in 0° /90° and 45° /45° . Firstly, a CNN-based semantic segmentation followed by a PCA-based registration is used to initialize the femur’s 3D pose and isotropic scale. Then, CNN-based regressors refine the 3D pose parameters. Then, the second stage deals with local 3D shape and 3D scale deformation by merging CNN-based local 3D displacement and local 3D scale ratio estimation of 17 handles with the Moving Least Square (MLS) deformation to get better fit to the patient’s radiographs. To validate the first stage, the 3D pose and isotropic scaling errors of the femur are validated in comparison to fuzzy gold standard personalized 3D models of the femur, reconstructed by an expert via a semi-automatic commercial software tool, SterEOS. In the second stage, the accuracy of local 3D shape, 3D scale, and the personalized 3D model of the whole femur is validated on two different validation sets. The first validation set includes 15 fuzzy gold standards of the personalized 3D models of the femur reconstructed via the semi-automatic SterEOS. The mean and standard deviations ሺmean±STDሻ of Root Mean Square of point-tosurface distance errors (RMS-P2S) is ሺ0.88±0.29ሻ mm. The second validation set comprises 5 gold standard personalized CT-scan-based reconstructed 3D model of the femur. The ሺmean±STDሻ of RMS-P2S errors is ሺ2.70±0.39ሻ mm.

Date	20 Apr 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Carlos Vázquez (Supervisor) & Jacques A. de Guise (Co-supervisor)