From segmentation to 3D insight

Abstract

The first step in all medical 3D modelling is image segmentation. Image segmentation is the identification and separation of objects of interest. This keynote will address the current state of image segmentation, key limitations, and future directions in image segmentation and 3D modelling. Despite the progress in image segmentation achieved with convolutional neural networks, major challenges still exist. Processing full-resolution CT image volumes is not possible on desktop graphics cards, and networks struggle in situations with large anatomical variability. These limitations restrict scalability and broader clinical adoption. A novel hybrid segmentation framework will be presented that addresses these limitations. The method combines a low-resolution convolutional neural network with random-walk image segmentation. This allows the use of global context while still achieving high precision, and enables the method to run on standard desktop graphics cards. 3D printed models are increasingly used clinically for anatomical visualization, surgical planning, and patient-specific guides. These applications have clear, proven clinical value, but they represent only a fraction of what is possible. In many applications, physical 3D printed models may not be required, as virtual reality could provide the necessary spatial understanding for surgical planning. Augmented reality is emerging as a powerful tool for intraoperative guidance. A central challenge, however, remains the accurate registration of digital 3D models to patient anatomy, particularly in soft tissue. Robotic surgery is an exciting new frontier. Current robotic surgery systems mainly act as extensions of the surgeon’s hands, but the next generation will aim for greater autonomy, including tasks such as drilling or cutting. These tasks will depend on detailed and accurate 3D models, increasing the demands on image segmentation methods. By advancing image segmentation, we can unlock new opportunities for 3D printing and 3D modelling, ranging from novel immersive visualization techniques to robotic systems, pushing the field beyond its current boundaries.