The boom in artificial intelligence and rapid deployment of AI-assisted tools is quickly seeing transformations across numerous fields, and with it, the challenges of adapting to this “new wave”. This year alone saw over 7000 new AI-assisted tools come online, so if you’re in search of a specific tool, it’s likely there’s already an AI for that. The big question in biomedical research is how AI will be integrated into our analyses, and how this will transform the pace and type of data that can be mined. One field that stands to strongly benefit from AI assistance is anatomic pathology, but how do the current AI tools hold up and how close are these to being deployed routinely for personalized medicine?
Traditional pathology has been the cornerstone of modern medicine and plays a pivotal role in diagnosis, effective treatment strategies and overall patient outcomes. The vantage point offered by microscopic evaluation of a tissue biopsy still outweighs bulk molecular profiling approaches which, when assessing a tumor prognosis for example, is paramount to patient care (sidenote – enter spatial biology which integrates both modalities and likely to be the future of modern pathology). Nevertheless, routine pathology remains a laborious and time-consuming endeavour requiring the collection and meticulous preparation of tissue samples for microscopic examination by a pathologist. Complicating matters further, the heterogeneous morphologies of tissues, the arbitrary selection of regions for detailed analysis, and the subjective nature of findings assessment often result in a low level of agreement among different pathologists regarding the diagnosis. This inherent complexity and subjectivity not only extend the duration of the diagnostic process but also underscores the need for innovative solutions like AI to enhance the accuracy and efficiency of pathology practices.
A Delphi study conducted by the Lancet this year gathered expert perspectives and expectations regarding the role of AI in anatomic pathology from those with first-hand computational pathology and AI experience. The consensus amongst these experts shows strong certainty that AI applications in pathology would become commonplace by 2030. Already, repetitive tasks that were traditionally very time consuming are already routine in pathology labs including automated QA/QC, stain normalization, ink dot removal, image data augmentation, object detection and segmentation, case prioritization, analytical (microorganism detection and tumour grading/measurement) and post-analytical tasks (enforcement of mandatory second reads upon significant discrepancies between pathologists). Pathologists were less confident regarding “aspirational” tasks, such as the identification and application of predictive signatures based on molecular profiles. This contrasts with academic researchers and industry stakeholders that had more confidence in the near-term role of AI for precision medicine. Nevertheless, there was a widespread agreement that AI can enhance diagnostic accuracy metrics. This improvement would be achieved, in part, by aiding in the identification of rare occurrences like small tumour clusters and metastases, standardizing the assessment and grading of tumours, and introducing more quantitative analyses in histopathology.
Currently, the most widely used algorithms in digital pathology are convolution neural networks (CNNS), which have emerged as state-of-the-art methods to solve real life computational pathology tasks. Like other supervised machine learning methods, CNNs are trained using a dataset with an expected outcome. These algorithms take digitalized pathological whole-slide images (WSI) as their input and acquire knowledge about correlations between specific attributes and specific labels, such as diagnoses assigned by pathologists, underlying molecular traits, and outcome metrics like patient survival or response to adjuvant/neoadjuvant treatments. Some of the current applications of CNNs in digital pathology include identification of tumour regions, prediction of survival based on tumour shape, grading prostate cancer and mitosis detection, with more algorithms being developed. AI-assisted pathology has already demonstrated its capability to enhance agreement among pathologists in various domains, including the evaluation of subjective characteristics like cytonuclear pleomorphism/degree of atypia, cellularity, mitotic figure quantification, scoring of tumour-infiltrating lymphocytes, and assessment of proliferation through the Ki67 index. Applications of AI in pathology diagnosis may also include rare and complex cases that are often challenging to general pathologists who rarely come across them. AI has the power of searching for images that are similar to query images from large histopathological databases, also known as content based image retrieval (CBIR).
Figure 1: Example of metastasis detection in the lymph node for a lung adenocarcinoma patient. (A) Hematoxylin and eosin (H&E)-stained lymph node pathology slide. Tumor cells began to invade into the capsule in the orange box, with white arrows pointing to tumor cells. (B) Cell classification result overlaid on the H&E image. Green: Tumor nuclei; blue: Lymphocytes; red: Stroma nuclei; cyan: Necrosis. Adapted from Cancers 2019, 11(11), 1673.
Simultaneously, these algorithms also possess the potential to extend their scope beyond the visual assessment of fundamental histopathological attributes to capture subtle patterns that are beyond human recognition, notably, assessing specific aspects of the tumour microenvironment (TME). AI can amalgamate information derived from TME analysis to formulate comprehensive diagnoses and correlate these attributes with other patient-related data, potentially providing valuable insights into disease progression, patient outcomes, or predictions regarding therapy response. Some successful applications of machine learning include the accurate grading of prostate cancer, identification of biomarkers for disease specific biomarkers in early-stage melanoma, detection of invasive breast cancer regions and prediction of response to chemoradiotherapy in rectal cancer. More studies are underway and likely to see major advances, however it should be noted that machine learning algorithms are reliant on high quality and accurate datasets in order to maximize utility.
Pushing capabilities: Deep learning in spatial omics
Tissue analysis is seeing a major shift in capabilities with high-content data more readily available. Various multiplexed platforms have been developed which allow the interrogation of multiple markers on a single section. Through these methods, a more holistic view of spatial relationships can be unveiled by determining the roles and interactions of the different cell types, both at the spatial and functional level. Whilst powerful, this kind of analysis is highly time consuming to perform and requires consistent and reproducible interpretation of large and complex datasets, a perfect showcase for AI algorithms. Unsurprisingly cancer research has seen the biggest boom in machine learning algorithms for incorporating high-content datasets for predictive medicine.
Cancer is a complex disease with varying aetiologies, molecular mechanisms, clinical characteristics, and prognosis. Imaged-based analysis of intra-tumour heterogeneity, spatial patterns of cell phenotype, and the interactions of these cells at the tumour interface have been shown to be useful for diagnosis and predictive indicators of therapy response. There is a large push in recent years to incorporate image analysis and AI methods to concurrently assess multiple biomarkers from different studies to provide improved diagnosis and selection of appropriate treatment regimens. An example of successful applications of spatial analysis for cancer is the assessment of the proximity and density of specific cells as a measure of immune activation. For example, lymphocyte activation gene 3 (LAG-3), which is expressed on exhausted T cells, has been shown to principally interacts with major histocompatibility-II molecules (MHC II) which are expressed on the surface of antigen-presenting cells. A spatial analysis study in bladder and gastric cancer showed that the density and proximity of LAG-3+ cells were significantly greater when associated with MHC II+ vs MHC II– tumour cells. This suggests that LAG-3 expressing tumour infiltrating lymphocytes (TILs) preferentially localizes to MHC II+ with proximity allowing engagement and activation of LAG-3. This information can provide information on immune response and activation and could be used to identify patient response. AI has also been used to quantify immune cells within the TME to define T-cell abundance and associated geographic localization in the tumour stroma, parenchyma, parenchyma-stromal interface, and invasive margin, which are then associated with factors to define underlying mechanisms of resistance to immunotherapy. A study investigating classic Hodgkin lymphoma (cHL) found that the cHL microenvironment is highly enriched for cytotoxic T lymphocyte-associated protein 4 (CTLA-4)-positive T cells that outnumber PD-1 positive and LAG-3 positive T cells. This implicates an immunologically privileged niche that is refractory to PD-1 blockade. These results suggest that anti–CTLA-4 or combination anti–CTLA-4 and anti–PD-1 might benefit a subset of patients with cHL, especially those unresponsive or resistant to anti–PD-1 alone. Studies have also used spatial analysis to determine the response of patients with non-small cell lung cancer (NSCLC) to nivolumab therapy. These included training machine learning models to extract morphological details, such as the spatial arrangement of tumour nuclei and variance in shape and chromatin structure, as well as the area and density of TILs and the proximity of TILs to each other and to tumour cells. The features extracted from these models were able to successfully distinguish patients who responded to nivolumab therapy. The application of new algorithms, and the refinements of existing ones, is currently underway as more datasets become publicly available and we are likely to see an explosion of computational tools for cancer precision medicine.
Figure 2: Deep learning method for standard H&E images for primary melanoma tumours to identify patients at risk for relapse. A raw H&E scan is first segmented and classified across three classes (tumour, immune and non-cell object). The total image is then split into tiles, a patch is drawn and thresholded for the presence of white space, tumour or immune objects. Patches with a high cell density are added to a deep neural network sequence and cell counts, cluster proportion and nuclear area are calculated. Each patch is then analysed through a 5-sequence CNN layer and run through a final fully connected layer. The output is a binary vote for the sequence of “recurrent” or “non-recurrent”. Votes are aggregated generate the subject’s recurrence prediction based on a majority vote. Adapted from Clin Cancer Res. 2020 Mar 1;26(5):1126-1134.
Challenges and Future Applications
This AI “gold rush” is both a blessing and a curse. Whilst the benefits of applying AI to anatomical pathology labs are clear, several challenges remain. Concerns relating to reproducibility, interpretability, accuracy and standardization between competing devices, high financial costs for processing hardware and a lack of regulatory guidelines have been barriers to large scale adoption for clinical utility. At present, although there are a few instances of AI being used in clinical trials, most of the AI applications have been observational.
Only a few AI/ML models have been properly validated and even fewer have become regulated products for routine use. The FDA, EMA and other regulatory bodies have kept a watchful eye on AI-based medical software assistance with an attempt to provide a regulatory framework to ensure rigour and consistency. Several professional and academic societies have developed recommendations for AI/ML applications in their respective fields; such as the Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement/the Prediction model Risk Of Bias Assessment Tool (PROBAST) for AI (TRIPOD-AI/ PROBAST-AI), Checklist for AI in Medical Imaging (CLAIM), Minimum information about clinical AI modeling (MI-CLAIM), DEcision support systems driven by AI (DECIDE-AI), checklist for AI in medical physics (CLAMP), and Standard Protocol Items: Recommendations for Interventional Trials-AI. The goal of these recommendations is to ensure rigorous and reproducible research of AI/ML in their respective fields. However, this may inadvertently lead to another dilemma regarding the consistency across these many checklists themselves and possibly the lack of standardization of these “standards”, further complicating the matter. A literature review on how to implement AI in healthcare practice concluded that this process is still in its early stages of development and that further research is needed to provide the knowledge necessary to develop implementation frameworks to guide the future application of AI in clinical practice.
Conclusions
The rapidly evolving field of digital pathology and AI-assisted tools has seen major breakthroughs with the potential to revolutionize precision medicine for oncology research. The applications are wide-ranging, from diagnosis and screening, prognosis, assessment of therapy response and applications for drug discovery and translational science. These can be considered a paradigm shift in how pathology services will be managed in the future, not only more efficient and rapid but also capable of personalization for patient-centred care. The development of appropriate tools requires input from multidisciplinary teams to provide efficient services that are clinically useful. Already, AI assistance provides tools for pathologists to streamline workflows and provide more accurate and consistent results. The development of new AI algorithms for diagnostic, prognostic/predictive and multi-modal dataset integration will lead to more advances for precision medicine in the future.
So, are robots poised to take over from pathologist? Not quite yet, instead human-AI combination will allow for faster, more accurate decision making in the clinic.
Article written by Caroline Beltran (not a robot)
References
- Araújo, T. et al. Classification of breast cancer histology images using Convolutional Neural Networks. PLoS One. (2017).
- Baxi, V. et al. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod. Pathol. (2022).
- Bulten, W. et al. Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard. Sci. Rep. (2019).
- Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med (2019).
- Cruz-Roa, A. et al. Accurate and reproducible invasive breast cancer detection in wholeslide images: a deep learning approach for quantifying tumor extent. Sci. Rep. (2017).
- El Naqa, I., et al. Translation of AI into oncology clinical practice. Oncogene (2023).
- Kulkarni, P. M. et al. Deep learning based on standard H&E images of primary melanoma tumors identifies patients at risk for visceral recurrence and death. Clin. Cancer Res. (2020).
- Niazi, M.K.K. et al. Digital pathology and artificial intelligence. Lancet Oncol. (2019).
- Patel, S.S. et al. The microenvironmental niche in classic Hodgkin lymphoma is enriched for CTLA-4-positive T cells that are PD-1-negative. Blood. (2019).
- Rakha EA, et al. Current and future applications of artificial intelligence in pathology: a clinical perspective. J Clin Pathol.(2021).
- Rojas, F. et al. Multiplex immunofluorescence and the digital image analysis workflow for evaluation of the tumor immune environment in translational research. Front. Oncol. (2022).
- Shmatko, A., et al. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. Nat Cancer 3, 1026–1038 (2022).
- Szabo, P.M. et al. CD8+ T cells in tumor parenchyma and stroma by image analysis (IA) and gene expression profiling (GEP): Potential biomarkers for immuno-oncology (I-O) therapy. Jour Clin Onc. (2019).
- Zhang, F. et al. Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features. Clin. Transl. Med. (2020).
- Zidane, M. et al. A review on deep learning applications in highly multiplexed tissue imaging data analysis. Front. Bioinform. (2023)
