Problem Description

Digital pathology has been gradually introduced in clinical practice. Although digital pathology scanners could give very high resolution whole-slide images (WSI) (up to 160nm per pixel), the manual analysis of WSI is still a time-consuming task for pathologists. Automatic analysis algorithms offer a way to reduce the burden for pathologists. As such, a large number of scientific papers on automatic image analysis of whole-slide images has been published recently and a few “Grand Challenges” were organized to evaluate algorithms for different diagnostic purposes. The first well-known challenges using WSIs in histopathology were CAMELYON16, TUPAC and CAMELYON17. The aim of these challenges was to detect the micro- and macro- metastases in lymph node in hematoxylin and eosin (H&E) stained WSIs (CAMELYON16/17) and to assess tumor proliferation in breast cancer (TUPAC). 

In 2010, the results from the largest randomized control lung screening trial, the National Lung Screening Trial (NLST) led to the implementation of lung cancer screening with low-dose Computed Tomography in the United States in 2015.  Recently, the results from the second largest randomized control trial, the Dutch-Belgian lung cancer screening trial (NELSON) also show the benefits of implementing lung cancer screening. The implementation in the U.S. and the possible implementation of lung cancer screening in Europe will likely lead to a substantial amount of whole-slide histopathology images biopsies and resected tumors, while the workload and the shortage of pathologists are severe. 

Our proposed challenge will focus on detecting and classifying lung cancer. This subject is of high clinical relevance because lung cancer is the top cause of cancer-related death in the world. At the same time, the number of qualified pathologists is too small to meet the huge clinical demands worldwide, especially in countries such as China with a big population of lung cancer patients. Automatic assessment of lung biopsies by an artificial intelligence (AI) system might help pathologists to reduce the workload and prevent from subjective bias in lung cancer diagnosis. Furthermore, these algorithms could train and assist inexperienced pathologists in the future.

Data Description

For a challenge to be successful, the obtained results should directly translate to the real world. This can only be ensured if the challenge data is representative of what is encountered in clinical practice. We propose a two-step strategy to maximize the likelihood of this challenge to reach that standard.

STAGE ONE: The first stage of the challenge will focus on detecting and segmenting lung carcinoma in WSI. For this stage, 200 H&E stained biopsy samples with cancer will be provided. All samples have been digitalized using the same scanner. Experienced pathologists have manually annotated the cancer regions on tissue level for each WSI. Original images and the annotation will be provided to all challenge participants.

STAGE TWO: The second stage of the challenge will focus on classifying the main lung cancer subtypes using WSI.

Evaluation Metrics

For the stage ONE, Dice coefficient will be used as the evaluation metric.

Test data is READY and submission is OPEN. The submit format could be found in "Submit" Section

Only the group who submit the result along with maximum 4 page paper (describing their methods ) will be counted as valid submission. Top 10 group will be invited to the ISBI challenge to share their work in Venice.

For the stage TWO, the classification accuracy will be the metric (will be started by the end of 2019).

Participation

Results in both stages uploaded to this website will be evaluated and the evaluation will be publicly available on the Results page. Participating teams have the full ownership and rights of their method. It is mandatory to submit a maximum 4 page paper describing the algorithms and results in detail (Deadline: Apr 5th 2019).

For each stage, the top 10 participating teams will be invited to contribute to a joint journal paper with maximum 2 authors per team describing the methods used and the results found in this challenge. The paper will be submitted to a high-impact journal in the field (e.g. IEEE TMI, Medical Image Analysis) no later than 8 months after each stage of the challenge.

Important Dates

Stage ONE: 100 training data release: Jan 5th 2019

Stage ONE: 50 training data release: Jan 15th 2019

Stage ONE: 50 test data release: Mar 4th 2019

Stage ONE: submission deadline: Mar 15th 2019  (24:00, pacific time zone, Mar 29th 2019)

Stage ONE: result pre-announcement: Mar 16th 2019 (Mar 30th 2019)

Stage ONE: paper submission: Apr 5th 2019 （24:00, pacific time zone, 4 page paper, IEEE double column format）

Stage ONE: official result announcement: Apr 11th 2019

Citation

Please reference the following papers if you use ACDC@LUNGHP data for a scientific publication:

Zhang Li et al.  "Computer-aided diagnosis of lung carcinoma using deep learning - a pilot study".

Zhang Li et al. “Deep Learning Methods for Lung Cancer Segmentation in Whole-slide Histopathology Images - the ACDC@LungHP Challenge 2019”

Sponsors

Special thanks to the following sponsors of this challenge: