Automated RANO solution for clinical trial

Challenge

The Response Assessment in Neuro-Oncology criteria (RANO) was identified in 2010 to establish trustworthy and widely used response criteria and clinical trial endpoints for CNS tumors. Those criteria were developed in part to address the difficulties with measuring lesions that were classified as “measurable” and “non-measurable” in the literature. Nowadays, most frequently, semi-automated RANO criteria are present in a clinical workflow. Although attempts to automate this process have been made, some of them have resulted in overestimated volumes. In one of our projects for a pharmaceutical company, we fully automated the identification and segmentation of various tumor areas for glioma patients both before and after surgery. A part of that challenge involved developing a trustworthy automated RANO algorithm.

 

What we did

We developed and followed a strict manual annotation procedure to guarantee the high quality of manual ground-truth delineations which were later used to build a deep learning segmentation model. The issue of significant inter-rater variability and disagreement among human readers for the bidimensional RANO calculation in the post-surgery images was lessened as a result of the end-to-end pipeline that was introduced.

We developed two solutions for RANO calculation:
  • The first one computes the product of the perpendicular diameters discovered through the analysis over all slices while adhering to strict RANO criteria.
  • The second one thoroughly optimizes the diameters’ product rather than their maximum length. This method, according to our research, is less sensitive to subtle changes and alterations in the contour of the lesions.

The result

In terms of consistency and accuracy, our automated RANO algorithm performed admirably when compared to the most-experienced expert readers (ICC: 0.681 and 0.866). We also demonstrated that tumor burden cannot always be determined solely by RANO measurements, hence investigating the automatically-extracted volumetric tumor characteristics may easily give more accurate insights about the disease progression. We believe that due to the automated tumor burden measurement’s high performance, it is possible to significantly enhance and streamline the radiological evaluation of glioblastoma in clinical trials and clinical practice. We also developed a new method of calculating RANO that is more resistant to poor automated segmentations and jagged contouring because it is less sensitive to small changes in the contour of the lesions.

Qualitative and quantitative analysis can reveal important aspects concerning the behavior and abilities of the suggested pipeline—an example axial post-contrast T1 image and manual segmentation (green—ET, yellow—ED, red—surgical cavity). Below, the RANO bidimensional measurements for the two most experienced readers compared with Automated RANO (Diameters) and Automated RANO (Product) obtained from either the ground-truth or the algorithm’s segmentation. Manual calculation of RANO is subjective, difficult to reproduce and may easily lead to high inter-rater disagreement—the proposed algorithm offers full reproducibility and a more accurate optimization of the product diameters leading to a stronger correlation with the ET volume.

Image comes from: Jakub Nalepa, Krzysztof Kotowski, Bartosz Machura, Szymon Adamski, Oskar Bozek, Bartosz Eksner, Bartosz Kokoszka, Tomasz Pekala, Mateusz Radom, Marek Strzelczak, Lukasz Zarudzki, Agata Krason, Filippo Arcadu, Jean Tessier, Deep learning automates bidimensional and volumetric tumor burden measurement from MRI in pre- and post-operative glioblastoma patients, Computers in Biology and Medicine, Volume 154, 2023, https://doi.org/10.1016/j.compbiomed.2023.106603

Our processing pipeline, alongside the experimental results were thoroughly discussed in the following paper:

Jakub Nalepa, Krzysztof Kotowski, Bartosz Machura, Szymon Adamski, Oskar Bozek, Bartosz Eksner, Bartosz Kokoszka, Tomasz Pekala, Mateusz Radom, Marek Strzelczak, Lukasz Zarudzki, Agata Krason, Filippo Arcadu, Jean Tessier, Deep learning automates bidimensional and volumetric tumor burden measurement from MRI in pre- and post-operative glioblastoma patients, Computers in Biology and Medicine, Volume 154, 2023, https://doi.org/10.1016/j.compbiomed.2023.106603