Kun-Hsing Yu, MD, PhD, is an associate professor of Biomedical Informatics at Harvard Medical School, an assistant professor of pathology at Brigham and Women's Hospital, and an instructor in epidemiology at Harvard T.H. Chan School of Public Health.

nterventional program specialist, chief of medicine and director of Gastroenterology and Endoscopy, City of Hope, Phoenix, and the founder and chair of the 

Endoscopic Oncology special interest group

Experts envision AI rapidly analyzing digitized pathology slides to assist diagnosis, highlight uncertainties, assess tumor heterogeneity, and support clinicians—enhancing accuracy while maintaining vital human collaboration.

The Future of Pathology with AI: Future Directions and Clinical Implementation

OncChats: The Future of Pathology with AI

, Toufic Kachaamy, MD, of City of Hope, Madappa Kundranda, MD, PhD, of Banner MD Anderson Cancer Center, and Kun-Hsing Yu, MD, PhD, of Harvard Medical School, envision AI rapidly analyzing digitized pathology slides to assist diagnosis, highlight uncertainties, assess tumor heterogeneity, and support clinicians—enhancing accuracy while maintaining vital human collaboration.

Can you walk us through the workflow that you have today, and what you imagine for the future? Biopsy was obtained, hematoxylin and eosin stains were done, and what happens? They're put in a scanner, and in 5 minutes, you get a report? Does it say 98% chance of this? If it's not one of the 120 cancer subtypes that you talk about, does it say unrecognized? Like for our viewers, can you explain how this [looks] today, and how you envision [this evolving] in the next 5 years?

The workflow we are envisioning is similar to how you proposed. Essentially, we still need to obtain informed consent and the sample from the patient, and then we can use standard scanners or any digitization method to [turn] the pathology samples into a machine-understandable, digital format. Once we have that, we can apply our published machine learning model to make the first-line screening and evaluation. The output of our model could be a summary report on the most likely diagnosis with the confidence level, or the level of probability that our machine believes the particular sample belongs to the different categories, and also the molecule-based prediction, as we discussed. For example, we can also predict the clinically actionable mutations and clinically actionable genomic variations related to specific treatment in order to inform the right clinical actions.

In addition to this, we're also thinking that we could use a user-friendly user interface to further identify the regions or machines leveraged to make the prediction. For example, if the diagnosis is colorectal adenocarcinoma, then we can first identify the regions that our machine used, that were used to inform, for example, the grade of the colorectal cancer, and also the regions that our machine used to determine microsatellite instability. Those may be the regions largely related to tumor-infiltrating lymphocytes. In addition to this, we can also have a section to further output the regions that our machine wasn't very sure about. And this could be important, because we know there could be a lot of autodistribution data points or some uncertainty even in expert-level diagnosis, so perhaps we can help save clinicians' time by automatically identifying the potential regions that require more extensive evaluation or even seeking a second opinion from a human expert. After all of this, we will be able to ensure that our machine will be able to facilitate a clinical diagnosis without missing some of the real cases or atypical manifestations that may merit additional review.

This is fascinating. One of the areas of my interest is human/machine interaction. We've seen in the gastroenterology world that sometimes AI improves outcome and sometimes AI worsens outcome; it depends on how the human/machine interaction happens, and [whether] the human has a positive view of AI or not. We are emotional beings at the end. From what I'm hearing from you, your system would highlight areas [indicating that we should] spend 5 minutes on this to get the diagnosis, or [specifying that it is] not certain [so we may] want to scrutinize this, instead of just spitting out [a diagnosis of] microsatellite-unstable adenocarcinoma. It's going to facilitate and make the pathologist way more effective in getting the diagnosis. Did I get that right?

Definitely. We are thinking that we shouldn't hide some of the intrinsic uncertainty in medical practice. Instead, our AI can quantify this level of uncertainty and seek a second opinion from expert clinicians.

: Dr Yu, this is fascinating, in so many different ways. We are literally starting to just scratch the surface as it pertains to AI and pathology. You've just done some real phenomenal work in the context of trying to move the field forward. One of the questions that I have for you, and you kind of alluded to this a few seconds ago, [pertains] to tumor heterogeneity. Can you just comment on that for a second? Because that's a huge part of it, as we look at it across the spectrum of oncology. How can AI and these predictive tools address that concern?

Definitely. So, as we know, most tumors are quite heterogeneous, and even for the same sample coming from the same patient, we can always see some levels of heterogeneity in terms of their microenvironment within the sample. Our approach, leveraging AI, is that we can even quantify this amount of heterogeneity. For example, we can quantify the amount of tumor-infiltrating lymphocyte in different regions and different microenvironments from the same sample.

With this information at hand, we can first quantify a heterogeneity score for each sample, and this may be useful for us to further interpret the genomic profiling results coming out of the sequences. And we can also use this information to guide the selection of treatments or even model and understand the evolution of the disease. In this case, we are thinking that our AI can augment many of these ongoing investigations on tumor heterogeneity by first quantifying this heterogeneity and investigating its association with clinical outcomes and other important variables.

: I couldn't agree more with that. For us in oncology, I'm dating myself, but almost like a decade and a half ago, molecular profiling came on board, and they said that it was the fourth tool that we had. Now, we have AI. But I think the key thing [is that we have] expert physicians, and expert pathologists, in this case, actually being able to review this. That's where the beauty of oncology is; it's truly a multidisciplinary sport. There is no way that we can get to a point wherein everything is done by the machine and there's no human interaction. I think that is the key part of it. And now, more than ever, this whole multidisciplinary approach and this collaboration has to be an integral part of what we can do to get the best outcomes for our patients. That's my takeaway from today. Your work has been phenomenal, and we really look forward to you doing a lot more, bigger and greater things, and continue to collaborate with all of us. Truly, thank you so much for your time today. This was very educational for me, and this would certainly be something that I'm looking forward to having ongoing conversations about.

Fantastic. Thank you so much for having me.

: Dr Kachaamy, any last words before we wrap up?

Yes, Dr Yu, I just want to say that your work is inspirational. It is showing us a glimpse at the future, and as Dr Kundranda said, we look forward to continuing this conversation and seeing where you will take us, and then help in making patients outcomes better.

Videos

Experts envision AI rapidly analyzing digitized pathology slides to assist diagnosis, highlight uncertainties, assess tumor heterogeneity, and more.

Experts discuss how AI analyzing pathology slides and clinical data can predict cancer prognosis and treatment response better than traditional models, with potential for clinical integration and AI tumor boards, though challenges remain.

The Future of Pathology with AI: Predicting Prognosis and Personalized Treatment

, Toufic Kachaamy, MD, of City of Hope; Madappa Kundranda, MD, PhD, of Banner MD Anderson Cancer Center; and Kun-Hsing Yu, MD, PhD, of Harvard Medical School, discuss how artificial intelligence (AI) analyzing pathology slides and clinical data can predict cancer prognosis and treatment response better than traditional models, with potential for clinical integration and AI tumor boards, though challenges remain.

I noticed in some of your research that you study prognosis. How are you using AI in pathology to predict prognosis? I really find that fascinating. Tell me more about that.

This is also one of the unexpected findings from our paper, in that we first built a standard diagnostic model out of hematoxylin and eosin [H&E] slides, and we further challenge the capability of such a large foundation model in predicting patient prognosis under standard treatment. Our approach is, in our study design, we first stratify the patients by their cancer types and cancer stages, making sure our patient subcohort receives more homogeneous type of treatment. Then we challenge our machine to make immediate predictions using the data we collected at the time of diagnosis—just a single H&E slide plus some basic demographics like patient age, patient biological sex, and many other factors. And we built a multimodality AI model that is incorporating different data modalities, different data types, all together into a clinical prediction.

We show that our model can reliably identify patients with high mortality risk [vs those] those with lower mortality risk, and we use the log-rank test to show the statistically significant difference between these 2 groups. The implication of this particular approach would be that in the future, when considering the standard treatment, perhaps we can first consult this AI model to make sure our patient will likely respond to the standard set of treatment; if not, we may consider more intensive clinical follow-up or...a more aggressive form of combination therapy.

I want to put this into perspective, especially as it relates to gastrointestinal malignancies. If we [consider] pancreatic cancer, [for example], a minority of patients will survive 5 years with standard treatment. But we expose a lot of patients to treatment, and we don't know how to predict these [cases]. Do you see AI, in the future, telling us, "This 10% is going to respond phenomenally to this treatment?" Is that correct? Because that would be phenomenal.

Yeah, totally. One of the potential uses of our approach [could be] to identify those exceptional responders, those who would respond to a particular treatment of interest. We can use this to personalize treatment decisions for individual patients.

 This kind of reminds me [that] every time you look at the bell-shaped curve, you look at 2 things as scientists. We look at the front end and the tail end; the ones who don't respond at all, and the ones who have amazing responses—as you mentioned, the ones who have a survival of 10 to 15 years. So far, we've been very rudimentary in terms of being able to predict what causes that. We have worked this up, ad nauseam.

I think the biggest advantage with AI...and please correct me [if I'm wrong], Dr Yu, is the fact that we have these large databases that also have these adaptive learning modules, and as we get more information, as we clean up the data, it basically gets better at predicting. That is intriguing for me, but it also leads to the next question, which is: How does this foundation model compare to the traditional models we have?

As we know, the foundation model is essentially a large model trained with a diverse and large amount of data. For example, ChatGPT is a foundation model with a large corpus of text information from Wikipedia and other available resources from the internet. Once this model is built, its capabilities are no longer restricted to answering some questions. We can also retrieve from Wikipedia. We can even use ChatGPT to plan for our next trip or to experiment around some creative recipes. Our idea here is very similar. In the past, standard pathology AI approaches, including many of those that we developed around 10 years ago, largely focused on a particular use case. For example, for colorectal cancer diagnosis, researchers just trained and optimized the particular model that is specialized for that disease distinction.

Our hypothesis here is that we can use a large and diverse amount of data and allow the machine to explore some of the data similarities and differences all on its own in order to learn the basic pathology knowledge about the general manifestations of different disease types. Building up on this hypothesis, we built this foundation model, which we call CHIEF—the Clinical Histopathology Imaging Evaluation Foundation model. In this CHIEF model, essentially, we first use a self-supervised approach to learn the similarities and differences among more than 15 median image patches and an additional 60,000 whole slide pathology images, and we learn their general pathology manifestation distributions across 19 different cancer types.

After this model is trained, we further tailor this model to differentiate different molecular variations, different survival outcomes specific to each disease entity. We show that this new approach, which leverages the overall data distribution, then, from a large and diverse amount of data, combined with weekly supervised machine learning that is tailored for specific tasks, worked much better than many of our early works in cancer-specific and use case–specific applications. This indicates that these AI approaches are very flexible. It can learn from a vast amount of data and capture some of the subtle details that may evade human eyes. With this capability, we can train this machine learning model to do something that is beyond human capability.

 You're spot-on, Dr Yu. I was in Mexico City last week, and for my plan of what I needed to do and where I needed to visit, I used ChatGPT. It gave me an amazing itinerary, and it was phenomenal. But besides that, I was very excited to look at your most recent publication and review that. I think this is where the road really meets the thing in the context of where we are going to be able to get all of this into the community.

We've looked at molecular tumor boards. To a certain degree, we're going to get to AI tumor boards, so to speak, and that is an exciting part of it. When we spoke about molecular tumor boards about 15 years ago, people were just laughing at us and saying, "No, this isn't even going to make sense." Now, everyone knows that even in the smallest community setting, there is some kind of molecular tumor board. But this AI tumor board, so to speak, when do you think we would get to that point?

Depending on our perspective, currently, there are actually a few FDA-approved pathology AI applications already out there, but those are largely application driven and application limited. For widespread, large-scale adoption, perhaps we still have some roadblocks that we need to overcome. How do we incorporate this new AI model into the current clinical workflow without disrupting our usual pathology and clinical evaluation? And how do we best utilize the information we extracted from this AI model, such that we wouldn't be misled by some imperfect predictions from these models? After overcoming this hurdle, I'm sure this AI technique could make a huge impact in the scenario of cancer diagnosis and the treatment selection and suggestions for clinicians.

The Future of Pathology With AI: Predicting Prognosis and Personalized Treatment

Experts discuss how AI can rapidly predict molecular subtypes from H&E slides with 87% accuracy, enhancing but not replacing genomic testing.

The Future of Pathology with AI: Molecular Subtyping and Genetic Testing

, Toufic Kachaamy, MD, of City of Hope; Madappa Kundranda, MD, PhD, of Banner MD Anderson Cancer Center; and Kun-Hsing Yu, MD, PhD, of Harvard Medical School, discuss how artificial intelligence (AI) can rapidly predict molecular subtypes from hematoxylin and eosin (H&E) slides with 87% accuracy, enhancing but not replacing genomic testing.

Let me just switch gears for a second. When we're looking at molecular subtyping of these pathology images, at some point, could AI replace genetic testing? Let me give you an example. When we look at the different molecular subtypes that we have in gastric cancer, or even microsatellite instability–high patients, how would AI factor into [this]? Right now, we are doing it through immunohistochemistry [IHC]—the basic [approach]—and then we are doing polymerase chain reaction [PCR]. Then there's a 5% discordance between PCR and doing IHC for [these patients], for example. How would AI factor into that?

One exciting and kind of surprising finding from our recent paper, [which was] published in 

, is that our foundation model can not only make traditional classifications based on pathology patterns, but we can also even predict their molecular subtypes and their molecule profiles just based on the standard H&E-stained slide. Then the natural next question is: How are we going to apply such a model? Will this completely replace the current genetic testing? Our thought is that perhaps this could be a useful tool to enhance some of the real-time genomic diagnosis.

For example, in certain cancer types, it's important to identify specific cancer subtypes with different treatments. Our AI model, operating on standard H&E-stained slides, will be able to identify patients with a high likelihood of harboring those crucial genomic variations. So, this could be a way to enhance the current genomic profiling process for [patients with] cancer. However, none of these models [are] perfect in identifying their molecular subtypes. As such, we don't think that, in the short run, our H&E AI-based approach would completely replace standard genomic profiling.

 What is the concordance and discordance that you currently have? What's the standard-of-care testing that we currently have?

For many of the important clinical genomic profiles—for example...as you mentioned, microsatellite instability–high and -stable status—we are able to achieve an accuracy of more than 87%. [That means that] more than 87% of the time, we're able to make an immediate prediction at the time of pathology diagnosis. But there's still 13% of the time where we are discordant, with the final truth coming from the sequencers. Those are the domains that we believe additional research on different AI models, and the recent breakthroughs in agentic AI and many other new AI systems, may be able to help.

That's fascinating. One of the issues is time to get the results of genomic testing. I assume the AI will give it immediately, or very quickly. Is that correct?

That's correct. We can optimize our model such that you will be able to provide diagnostic and molecular predictions within a second. This is quite different from the traditional approach, where we may have to wait for days, [or] even weeks, to get genomic results.

Have you been successful in, for example, getting a targeted agent approved based on your results, or is it still in the research realm?

Yeah, we are currently partnering with a few pharmaceutical companies to work with early-stage trials, and we build our AI system as a part of companion diagnostics identifying the patient who would likely be responsive to the particular drug under investigation. We feel that this may be a fast track toward embedding our AI system into clinical practice.

I assume that we're only improving with time, right? If you have 87% concordance today, with time, it's going to be better given the fast improvement in this technology. This is fascinating.

The Future of Pathology With AI: Molecular Subtyping and Genetic Testing

Experts discuss how AI can diagnose cancer from standard H&E slides while accounting for slide quality differences to support broader clinical use.

The Future of Pathology with AI: Practical Applications

, Toufic Kachaamy, MD, of City of Hope; Madappa Kundranda, MD, PhD, of Banner MD Anderson Cancer Center; and Kun-Hsing Yu, MD, PhD, of Harvard Medical School, discuss how artificial intelligence (AI) can diagnose cancer from standard hematoxylin and eosin (H&E) slides while accounting for slide quality differences to support broader clinical use.

Can you tell us, on the practical side, how this looks in everyday life? Is there a specific staining that needs to happen? Is this for any staining? How quickly [does it] happen? [Do] the pathologists have to assign this and review? Just explain to us: How does this work in real life?

In our current research, we are primarily working with the standard hematoxylin and eosin staining slides because those are the most common and standard for diagnosing most types of cancers. We've been fortunate to have lots of collaboration with many hospitals worldwide to collect diverse and large data sets of the standard H&E pathology. We show that using AI and, more specifically, computer vision approaches involving vision transformers, we are able to train a model that will be able to differentiate and diagnose different types of cancers using standard H&E. You also make a good point that there are a few special stains that clinicians and pathologists use to further identify the molecular subtypes, and we further show that our method is applicable to those extensions, as well [as] to analyzing [immunohistochemistry and] many other special stains specific for individual cancer types.

Dr Yu, I think this is fantastic. What you're even alluding to is truly a pipe dream. Because when we look at cancer, 70% [or more of cases are] actually managed in the community. It's less than 25% to 30% [of cases that are] managed in tertiary care centers. I think this is going to be the key context [to this]. Going back to Dr Kachaamy's question, [which was related to] the workflow. [Regarding] the H&E stains, unfortunately, as a grad student, I've done my piece of that. Luckily, I don't need to do that anymore. But I think the practical part of it is also the quality of the specimens in the community, based on the expertise [and] the technology that they have.

I'm a gastrointestinal medical oncologist, so I'm just going to keep it simple. Even when I get a fine-needle aspiration or a fine-needle biopsy of the pancreas from Dr Kachaamy, the quality is very different from another advanced endoscopist in the community. It might be because of expertise, but it could be because of the equipment and the needles that they're using. So, my question is: When we're trying to translate this to the real world, how is the quality of the specimen acquisition for these H&E slides a part of this AI algorithm? Because you're going to have a spectrum. What are your thoughts on that?

That's a great point. We acknowledge that for many different slides [that] we obtain from different sources, they do have variable qualities, and that presents a big hurdle, even for expert human evaluators. Occasionally, our expert pathologist collaborators will be able to immediately tell whether a particular slide is from their hospital or from outside hospitals. Our approach to address this using this AI method is that we have image preprocessing modules that first detect the slide quality, and we have image augmentation approaches to ensure that in our training data set, we have augmented the data such that you are better able to capture a wide diversity and variety of possible tissue quality.

With this approach, we are able to extend and improve the generalizability of our model to further enhance its applicability into the clinical workflow. Both of you have identified this very important current hurdle, which is: How do we translate this important clinical research question into actual clinical practice? In my opinion, there are still a few ongoing challenges that we need to address as a team involving clinicians, AI developers, researchers, pathologists, and many others to further identify the right workflow to embed the current state-of-the-art AI system into day-to-day clinical practice.

 Thank you so much. I think that is such an important part of it, because just going to rudimentary pathology, there were some next-generation sequencing companies that used microdissection; this [was] almost a decade ago. That kind of made the [go-to] company only because, within pancreatic cancer [and] with microdissection, I was able to get rid of the stroma as opposed to actual cancer cells. That made a huge difference way back then.

Now, when you are referring to imaging augmentation and trying to get that purity aspect of it—as it pertains to diagnosis, prognosis, all of that—within the AI models, I think that is what is fascinating. That's what's exciting. That truly is the way of the future, because we cannot have our patients undergo multiple repeated biopsies. We need to look at it on the tail end to try to figure out how best we can actually improve what we have in terms of our technology, with the specimens, [and] with the slides that we have. We all know [that] even the H&E slides can vary from institution to institution based on the way it is stained.

The Future of Pathology With AI: Practical Applications

Experts provide an overview of AI's integration into modern pathology and the development of foundation models for cancer diagnosis.

The Future of Pathology with AI: Setting the Stage

, Toufic Kachaamy, MD, of City of Hope; Madappa Kundranda, MD, PhD, of Banner MD Anderson Cancer Center; and Kun-Hsing Yu, MD, PhD, of Harvard Medical School, provide an overview of the integration of artificial intelligence (AI) into modern pathology and the development of foundation models for cancer diagnosis.

Thank you all for joining us on this new series of 

. I am Toufic Kachaamy, MD, chief of medicine at City of Hope in Phoenix, [Arizona]. My cohost is Madappa Kundranda, MD, PhD, who is the chief of cancer medicine at Banner MD Anderson in Phoenix. And we are excited to be joined by Kun-Hsing Yu, MD, PhD, today, who is an [associate] professor in the Department of Biomedical Informatics at Harvard Medical School. Dr Yu developed the first fully automated artificial intelligence [AI] algorithm to extract thousands of features from whole slide histopathology images. He discovered the molecular mechanisms underpinning the microscopic phenotypes of tumor cells and successfully identified previously unknown cellular morphologies associated with patient prognosis. His lab integrates cancer patient multiomics, genomics, epigenomics, transcriptomics, and proteomics profiles with quantitative histopathology patterns to predict their clinical phenotypes.

And if you feel that you're lost with everything that I talked about, continue [to listen], because he will explain all of this to us. Dr Yu, thank you for joining us. Can we start by you just giving us an overview of where pathology is today and how AI is being integrated? [Let's unpack this] before we jump to the future, which is really [where] your work [is focused].

Thank you so much for having me, Dr Kachaamy. As you know, my lab is primarily focusing on developing novel AI approaches to analyze pathology data to enhance our current clinical practice. We have developed several foundation models that leverage a large and diverse amount of data to teach our machines how to recognize the basic pathology patterns related to diagnosis and prognosis. We've been fortunate to be successful in applying such platforms to many different clinical use cases, including a few prospective studies that are currently ongoing in our hospitals.

Thank you for this clarification, Dr. Yu. Now, my first question to you is: Can AI diagnose cancers using standard pathology slides today?

That's a great question. In the past few decades, many researchers have been trying to develop quantitative approaches to analyze the whole slide digital pathology images. In our group, we show that using our recent foundation model approach, we can accurately diagnose more than 19 different types of cancers, including identifying the parts that were occupied by cancer cells from the adjacent benign tissue, as well as classify different types and subtypes of cancers that may have different clinical and treatment implications. We show that our diagnosis accuracy is on par with expert pathology evaluation. This indicated that as long as we are training the AI model using the right amount of data and with the right architecture, we can build expert AI systems that can recapitulate important diagnostic signals from standard histopathology slides.

Kun-Hsing Yu, MD, PhD

Articles

The Future of Pathology with AI: Future Directions and Clinical Implementation

The Future of Pathology With AI: Predicting Prognosis and Personalized Treatment

The Future of Pathology With AI: Molecular Subtyping and Genetic Testing

The Future of Pathology With AI: Practical Applications

The Future of Pathology With AI: Setting the Stage