Road to Automatic Hip and Femur Segmentation from CT

 

This use case describes the roadmap to create an AI model for fully automatic segmentation of hip and femur anatomy from CT.

Introduction

For modern surgical planning, for example of the hip, pelvis or femur, CAD modeling plays an essential role. Especially in the case of custom-made implants, which are tailored exactly to the patient's anatomy.

Before any patient-based CAD processing can begin, it is necessary to create the model itself. For this purpose, the anatomical target structures (hip, pelvis, ...) need to be segmented from the patient's CT images, using semi-automatic software tools as standard.

Especially in the case of individual anatomical or pathological conditions, such as pre-existing implants, various fractures or an osteosarcoma, existing brush annotation tools or threshold-based segmentations reach their limits.

This makes this preprocessing step time-consuming and costly.

Manual, pixel-accurate annotation of the left hip scoop surrounded by cancer using Chimaera Smart Brush Annotation.

Using the case study of automatic hip and femur segmentation, this blog post aims to show how the path to creating an AI model unfolds and what prerequisites must be met.

Through precise engineering work, a fully automated AI model can thus be integrated into the client's workflow, which performs a complete segmentation of the pelvis, femur, and implants in a matter of seconds.

Methods and Workflow

There is very rarely the ideal out-of-the-box AI model that is perfectly suited to the problem at hand.

Several steps are necessary before an AI model can automate the target task of segmenting pelvic blades and femurs in the daily work process.

First, the algorithm is specifically tailored to the customer's problem (which anatomical regions, which implants, specific products or manufacturers, etc.). Second, due to data protection and customer agreements, an AI model trained on client A's database cannot be distributed to client B.

Workflow steps for the creation of an AI model.

The path until the AI model generates segmentations for the customer at the push of a button can generally be divided into 4 stages.

Basically, a suitable stock of CT images and associated data annotations (manual segmentations) is required. On this basis the actual engineering takes place, whereby the algorithm is iteratively taught to learn the task on the basis of the existing data stock, to carry out a fully automatic segmentation with new image data. The goal of the integration phase is to prepare the AI model so that it can be seamlessly integrated into the customer's processing procedures.

At Chimaera, we also make it a point not to sell black-box solutions: Through training, source code and know-how transfer, we enable our customers to develop independently and implement future adaptations.

All these steps are provided by Chimaera's services from a single source and can thus be optimally coordinated.

Data Management


As a first step, the exact task should be defined in close exchange with the customer. In this example, the automatic segmentation of 4 classes is required:

  • left acetabulum
  • right acetabulum
  • left femur
  • right femur

If the task were extended to cases of hip revision, the existing implants would also have to be segmented and additional classes defined. In that case, it would be conceivable to keep these as one universal class (e.g. implant) or to make a more precise breakdown (e.g. cup, stem, screws, etc.) - depending on the application and the client's idea.

Annotation


A complete annotation of the defined classes is required for each individual data set, which serves as input for the AI during the training phase and enables it to detect shapes and structures in the CT scans.

High quality annotations are essential, as the AI algorithm interprets the pixel-precise markings as ground truth. The results of the final AI solution are expected to be only be as good as the preliminary work done in the annotations.

The segmentations performed by the client over years can also often serve as annotations. If this is not the case or only insufficient, Chimaera offers Annotation as a service by our medically trained team. High quality is guaranteed by the use of our specially developed software and the integrated quality process.

In general, the more complex the task and the higher the desired accuracy, the more data is needed to train the AI model. The range of data required can be between 100 - 10000 records. In addition, care must be taken to ensure that the variable parameters such as gender, population, pathologies, implants, etc. are as broad as possible so that the AI model can learn rules that are as generic as possible.

AI Engineering


As soon as initial data and annotations are available, an AI prototype can already be trained.

The first prototype will not yet deliver perfect segmentations, but it will be possible to identify tendencies for further optimization (network architecture, preprocessing, errors in the annotation, etc.). In addition, the finer points can be worked on in close exchange with the customer and his application expertise.

Even after this, the creation of the AI model remains an interactive process until the final algorithm finally emerges. And even here, regulatory requirements may require constant monitoring and improvement.

If necessary (data protection), all training can also be carried out directly at the customer's site.

Integration and Deployment


If the AI delivers the segmentations of the hip and femur in the desired quality, the algorithm can be integrated into the customer's work processes.

In its simplest form, this processing step can run on the work computer as a stand-alone process that reads CT data sets and stores the finished segmentation at a defined path.

Alternatively, the AI model can be integrated directly into an existing software solution.

Regardless of the type of integration, Chimaera delivers the full source code for the AI model to the customer. Likewise, the know-how of the new technology is passed on to the staff, so that independent further developments of the algorithm are possible.

 

Result

Depending on the complexity, a precise AI model can thus be developed in a few working weeks that segments acetabular cups and femurs fully automatically, as in the example.

The model can additionally be trained for new occurrences at a later time. Thus, in the case of implant detection, new manufacturers or models could be easily handled.

The finished AI model can then be used to segment anatomical structures from a 3D CT series fully automatically.