The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI. - Publication

Publication

The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI.

Journal Paper/Review - Jun 17, 2024

Moawad AhW, Janas Anastasia, Baid Ujjwal, Ramakrishnan Divya, Saluja Rachit, Ashraf Nader, Jekel Leon, Amiruddin Raisa, Adewole Maruf, Albrecht Jake, Anazodo Udunna, Aneja Sanjay, Anwar Syed Muhammad, Bergquist Timothy, Calabrese Evan, Chiang Veronica, Chung Verena, Conte Gian Marco Marco, Dako Farouk, Eddy James, Ezhov Ivan, Familiar Ariana, Farahani Keyvan, Iglesias Juan Eugenio, Jiang Zhifan, Johanson Elaine, Kazerooni Anahita Fathi, Kofler Florian, Krantchev Kiril, LaBella Dominic, Van Leemput Koen, Li Hongwei Bran, Linguraru Marius George, Link Katherine E, Liu Xinyang, Maleki Nazanin, Meier Zeke, Menze BjoeH, Moy Harrison, Osenberg Klara, Piraud Marie, Reitman Zachary, Shinohara Russel Takeshi, Tahon Nourel Hoda, Nada Ayman, Velichko Yuri S, Wang Chunhao, Wiestler Benedikt, Wiggins Walter, Shafique Umber, Willms Klara, Avesta Arman, Bousabarah Khaled, Chakrabarty Satrajit, Gennaro Nicolo, Holler Wolfgang, Kaur Manpreet, LaMontagne Pamela, Lin MingDe, Lost Jan, Marcus Daniel S, Maresca Ryan, Merkaj Sarah, Nada Ayaman, Pedersen Gabriel Cassinelli, von Reppert Marc, Sotiras Aristeidis, Teytelboym Oleg, Tillmans Niklas, Westerhoff Malte, Youssef Ayda, Godfrey Devon, Floyd Scott, Rauschecker Andreas, Villanueva-Meyer Javier, Pflüger Irada, Cho Jaeyoung, Bendszus Martin, Brugnara Gianluca, Cramer Justin, Perez-Carillo Gloria J Guzman, Johnson Derek R, Kam Anthony, Kwan Benjamin Yin Ming, Lai Lillian, Lall Neil U, Memon Fatima, Patro Satya Narayana, Petrovic Bojan, So Tiffany Y, Thompson Gerard, Wu Lei, Schrickel E Brooke, Bansal Anu, Barkhof Frederik, Besada Cristina, Chu Sammy, Druzgal Jason, Dusoi Alexandru, Farage Luciano, Feltrin Fabricio, Fong Amy, Fung Steve H, Gray R, Ikuta Ichiro, Iv Michael, Postma Alida A, Mahajan Amit, Joyner David, Krumpelman Chase, Letourneau-Guillon Laurent, Lincoln Christie M, Maros Mate E, Miller Elka, Morón Fanny, Nimchinsky Esther A, Ozsarlak Ozkan, Patel Uresh, Rohatgi Saurabh, Saha Atin, Sayah Anousheh, Schwartz Eric D, Shih Robert, Shiroishi Mark S, Small Juan E, Tanwar Manoj, Valerie Jewels, Weinberg Brent D, White Matthew L, Young Robert, Zohrabian Vahe M, Azizova Aynur, Brüßeler Melanie Maria Theresa, Fehringer Pascal, Ghonim Mohanad, Ghonim Mohamed, Gkampenis Athanasios, Okar Abdullah, Pasquini Luca, Sharifi Yasaman, Singh Gagandeep, Sollmann Nico, Soumala Theodora, Taherzadeh Mahsa, Yordanov Nikolay, Vollmuth Philipp, Foltyn-Dumitru Martha, Malhotra Ajay, Abayazeed Aly H, Dellepiane Francesco, Lohmann Philipp, Pérez-García Víctor M, Elhalawani Hesham, Al-Rubaiey Sanaria, Armindo Rui Duarte, Ashraf Kholod, Asla Moamen M, Badawy Mohamed, Bisschop Jeroen, Lomer Nima Broomand, Bukatz Jan, Chen Jim, Cimflova Petra, Corr Felix, Crawley Alexis, Deptula Lisa, Elakhdar Tasneem, Shawali Islam H, Faghani Shahriar, Frick Alexandra, Gulati Vaibhav, Haider Muhammad Ammar, Hierro Fátima, Dahl Rasmus Holmboe, Jacobs Sarah Maria, Hsieh Kuang-Chun Jim, Kandemirli Sedat G, Kersting Katharina, Kida Laura, Kollia Sofia, Koukoulithras Ioannis, Li Xiao, Abouelatta Ahmed, Mansour Aya, Maria-Zamfirescu Ruxandra-Catrinel, Marsiglia Marcela, Mateo-Camacho Yohana Sarahi, McArthur Mark, McDonnell Olivia, McHugh Maire, Moassefi Mana, Morsi Samah Mostafa, Muntenu Alexander, Nandolia Khanak K, Naqvi Syed Raza, Nikanpour Yalda, Alnoury Mostafa, Nouh Abdullah MohaAly, Pappafava Francesca, Patel Markand D, Petrucci Samantha, Rawie Eric, Raymond Scott, Roohani Borna, Sabouhi Sadeq, Sanchez-Garcia Laura M, Shaked Zoe, Suthar Pokhraj P, Altes Talissa, Isufi Edvin, Dhermesh Yaseen, Gass Jaime, Thacker Jonathan, Tarabishy Abdul Rahman, Turner Benjamin, Vacca Sebastiano, Vilanilam George K, Warren Daniel, Weiss David, Willms Klara, Worede Fikadu, Yousry Sara, Lerebo Wondwossen, Aristizabal Alejandro, Karargyris Alexandros, Kassem Hasan, Pati Sarthak, Sheller Micah, Bakas Spyridon, Rudie Jeffrey D, Aboian Mariam

Citation

Moawad A, Janas A, Baid U, Ramakrishnan D, Saluja R, Ashraf N, Jekel L, Amiruddin R, Adewole M, Albrecht J, Anazodo U, Aneja S, Anwar S, Bergquist T, Calabrese E, Chiang V, Chung V, Conte G, Dako F, Eddy J, Ezhov I, Familiar A, Farahani K, Iglesias J, Jiang Z, Johanson E, Kazerooni A, Kofler F, Krantchev K, LaBella D, Van Leemput K, Li H, Linguraru M, Link K, Liu X, Maleki N, Meier Z, Menze B, Moy H, Osenberg K, Piraud M, Reitman Z, Shinohara R, Tahon N, Nada A, Velichko Y, Wang C, Wiestler B, Wiggins W, Shafique U, Willms K, Avesta A, Bousabarah K, Chakrabarty S, Gennaro N, Holler W, Kaur M, LaMontagne P, Lin M, Lost J, Marcus D, Maresca R, Merkaj S, Nada A, Pedersen G, von Reppert M, Sotiras A, Teytelboym O, Tillmans N, Westerhoff M, Youssef A, Godfrey D, Floyd S, Rauschecker A, Villanueva-Meyer J, Pflüger I, Cho J, Bendszus M, Brugnara G, Cramer J, Perez-Carillo G, Johnson D, Kam A, Kwan B, Lai L, Lall N, Memon F, Patro S, Petrovic B, So T, Thompson G, Wu L, Schrickel E, Bansal A, Barkhof F, Besada C, Chu S, Druzgal J, Dusoi A, Farage L, Feltrin F, Fong A, Fung S, Gray R, Ikuta I, Iv M, Postma A, Mahajan A, Joyner D, Krumpelman C, Letourneau-Guillon L, Lincoln C, Maros M, Miller E, Morón F, Nimchinsky E, Ozsarlak O, Patel U, Rohatgi S, Saha A, Sayah A, Schwartz E, Shih R, Shiroishi M, Small J, Tanwar M, Valerie J, Weinberg B, White M, Young R, Zohrabian V, Azizova A, Brüßeler M, Fehringer P, Ghonim M, Ghonim M, Gkampenis A, Okar A, Pasquini L, Sharifi Y, Singh G, Sollmann N, Soumala T, Taherzadeh M, Yordanov N, Vollmuth P, Foltyn-Dumitru M, Malhotra A, Abayazeed A, Dellepiane F, Lohmann P, Pérez-García V, Elhalawani H, Al-Rubaiey S, Armindo R, Ashraf K, Asla M, Badawy M, Bisschop J, Lomer N, Bukatz J, Chen J, Cimflova P, Corr F, Crawley A, Deptula L, Elakhdar T, Shawali I, Faghani S, Frick A, Gulati V, Haider M, Hierro F, Dahl R, Jacobs S, Hsieh K, Kandemirli S, Kersting K, Kida L, Kollia S, Koukoulithras I, Li X, Abouelatta A, Mansour A, Maria-Zamfirescu R, Marsiglia M, Mateo-Camacho Y, McArthur M, McDonnell O, McHugh M, Moassefi M, Morsi S, Muntenu A, Nandolia K, Naqvi S, Nikanpour Y, Alnoury M, Nouh A, Pappafava F, Patel M, Petrucci S, Rawie E, Raymond S, Roohani B, Sabouhi S, Sanchez-Garcia L, Shaked Z, Suthar P, Altes T, Isufi E, Dhermesh Y, Gass J, Thacker J, Tarabishy A, Turner B, Vacca S, Vilanilam G, Warren D, Weiss D, Willms K, Worede F, Yousry S, Lerebo W, Aristizabal A, Karargyris A, Kassem H, Pati S, Sheller M, Bakas S, Rudie J, Aboian M. The Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI. ArXiv 2024

Type

Journal Paper/Review (English)

Journal

ArXiv 2024

Publication Date

Jun 17, 2024

Issn Electronic

2331-8422

Brief description/objective

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.