This video can't play due to privacy settings
To change your settings, select the "Cookie Preferences" link in the footer and opt in to "Advertising Cookies."
Driving healthcare discoveries with AI ft. Jianying Hu
How is artificial intelligence moving beyond generating text and art to tackle our most fundamental challenges in medicine? Explore the new frontier of AI in healthcare and the progress in the creation of domain-specific foundation models as Red Hat CTO Chris Wright is joined by Dr. Jianying Hu, IBM Fellow and Global Science Leader of AI for Health. They discuss the shift from general-purpose LLMs to models that understand the complex language of biology, simulating drug efficacy, validating new research, and enabling personalized medicine. Discover why the future of medicine isn't just about a single AI model, but about building an open, iterative ecosystem that combines deep domain expertise with the power of AI to understand biology from the ground up.
자막
Transcript
00:00 - Chris WrightWe often hear about AI's exciting potential to generate text or code or art. But today let's explore a different frontier, how AI is being used to tackle some of our most fundamental challenges in disease and medicine. We'll discuss how open innovation is fueling big changes in healthcare and what it takes to move these breakthroughs from the lab to create real-world impact. And to talk about that, we have Jianying Hu, IBM Fellow, Global Science Leader of AI for Health, and Director of Healthcare and Life Sciences Research at IBM. Welcome to Technically Speaking, where we explore how open source is shaping the future of technology. I'm your host, Chris Wright. Jianying, thanks for joining me.
00:45 - Jianying Hu
Thank you, Chris, really exciting to be here.
00:47 - Chris Wright
So you've been working in the space of AI and healthcare for quite some time. This is a new area for me, and I think for all of us, it's exciting because we can understand the potential impacts, but what drew you in and where did you see these intersections between healthcare needs and the potential of technology like ML and AI?
01:09 - Jianying Hu
Yeah, so my background is in machine learning. I started my research career at Bell Labs when I joined their multimedia research department, fresh out of graduate school. And then I started working on handwriting recognition. And then that work expanded into web content analysis and in general, just multimedia content analysis, including image and video. And then I joined IBM Research in 2003 where I was, you know, really attracted by the opportunity to get more real world impact from my research work. So I started at the math department really applying machine learning to business analytics, working on things like portfolio analysis and resource demand forecasting. Then around 2010, you know, driven by a desire to now really see more of a impact, positive impact on society coming from machine learning and AI research, I started along with a few other researchers at IBM Research to explore, you know, application of AI and machine learning to healthcare analytics. And once we started looking into that area, it became very quickly evident to us that there were a lot of really untapped opportunities there. So we decided to really devote ourself into this space. And that's the area that I have been focusing on since then. We believe AI is really, the impact really goes beyond code and creative work. It can really have a huge impact. It's already starting to impact, to help us solve, you know, complex challenges in this healthcare life sciences space. Probably the most famous example is AlphaFold, which, you know, shared the Nobel Prize for chemistry last year. But it's not just about these single breakthroughs or new models, it's really shifting the way we do science, right? So these new wave of AI models is allowing us to, you know, with the increasingly powerful large language models, the reasoning engines, combining that with domain-specific foundation models and also increasingly tools to create agentic workflows. AI is really allowing us to drive tighter integration of data coming from live experiments, coming from knowledge sources, coming from computational models and simulations as well as clinical encounters. And through that, it's making the biomedical research much more iterative, much more responsive and better aligned with real world use cases.
04:13 - Chris Wright
I think many of us have a better understanding of how generative AI works with LLMs, but underneath there's deep learning and neural networks, and a lot of the core technologies are actually the same. How do, what are you drawing from in terms of your focus in, say, drug discovery? What core tech underlying technologies that are maybe shared are you drawing from?
04:39 - Jianying Hu
Yeah, so I think the way that AI is going to drive faster and better discovery within healthcare life sciences space, particularly for biomedical research, is really this potential for domain-specific AI foundation models to be able to learn from rich multimodal data, a representation that can really capture the intrinsic patterns of biomolecular interaction and sort of capture the, how the human physiological system really works. And from there get deeper understanding. And so when we think about how to really arrive at such a rich representation, right? This is sometimes people refer to this as virtual cell, something that's so intrinsic that you can use it to drive many, many different downstream tasks. It's a very grand vision. It's something that has been attempted at before, but this time around what's really different is again, the availability of this AI foundation modeling technology. And so we have been really looking at how to build such a biomedical foundation model for, specifically for the purpose of driving biomedical research, right? And we very quickly realized you really need to go beyond language models in order to, you need to really build multimodal models to be able to capture the very rich raw data, if you will, that comes from many, many different biomolecular entities within this space. And so that's really, I think fundamentally where the technology challenge is in being able to use AI technologies to drive impacting biomedical research. And that's where a lot of the innovations took place.
06:37 - Chris Wright
So you're building in other contexts, we might call it a worldview, and that's foundational. It's built from primitives that represent the real world. This is not a simulated game world. This is--
06:51 - Jianying Hu
Right, exactly.
06:52 - Chris Wright
The real world with our biology. And then you're using that to do modeling and simulations that can help discover drugs or look at side effects and really increase that--
07:05 - Jianying Hu
Yeah, exactly.
07:06 - Chris Wright
Speed of discovery and delivery.
07:06 - Jianying Hu
Exactly, so to that end, we have, you know, created this family of biomedical foundation models to allow us to really tap into this rich data source going from multiomics to biomolecular data to lab essays, right, to get to these rich representations. And specifically, and a lot of the effort in building this family of, you know, biomedical foundation models is really to explore ways to allow us to inject ingest data from different modalities and to enable learning across many different domains, and many different tasks. So to that end, we have created really four groups of models within this family of biomedical foundation models. So with one group, which we call MMELON, is where we focused on exploring, ingesting, and integrating different views of molecules. So a sequence view, a imaging view, which is the image of the, really the drawing of the molecule. And then also graph view where a node is an atom and an edge can represent the bond between the atoms, right? And think about how to ingest these different views, and then to integrate them to get to a more intrinsic representation. In another group of models called MAMMAL, our focus was now to look at how do you enable learning across multiple domains, over a wide range of tasks, going from small molecule therapies to biologics, to, you know, multiomics DNAs and RNAs and how to represent that in a uniform way, in this case using sequence-based representation. And then the goal is to, through this multitask learning to see if we can leverage the relationships and interactions between these different domains. And then with the most recent BMFM-RNA and BMFM-DNA models, these are our omics models. We really focused on creating a very flexible software package that would allow us to have a configurable way of exploring different ways of pre-training, different ways of encoding and tokenization to accommodate these different types of data. And then also to run in a systematic way against different benchmarks and to have also interpretability models to allow us to really interrogate embeddings that have been learned from these validation models. So it's a very, broad range of technical challenges to tackle.
09:53 - Chris Wright
And I can imagine, I mean, you're benefiting from a lot of the work that's happening in the large language model space, not only directly because you can use large language models connected with your work, but also the, some of the observability techniques or interpretability techniques.
10:11 - Jianying Hu
Yes.
10:11 - Chris Wright
I imagine are applicable. How do you draw from other communities' efforts that are not focused in the medical research area?
10:21 - Jianying Hu
Yeah, so in many ways, right. So, that we are really learning and leveraging the success from large language model, but then making adjustments so that we can build models specifically for this domain. So for example, if we talk about tokenization schemes, right? So this is about one of the important things is really understand your data, understanding the nature of the data so that you can figure out what's the best way to come up with a representation and tokenization. So, one, in this space, some of the innovations that we have to make, besides just borrowing from the large language model approach of, you know, tokenizing your characters and words into these sequences is, for example, in the case of MAMMAL, like I mentioned, we wanted to be able to have a uniform sequence-based representation representing really entities from multiple domains. So in order to do that, we came up with a new way, new method of tokenization, that we call structured prompt syntax that allow us to really, with a structured syntax, to incorporate these different entities into the uniform sequence so that we can then use the standard technologies and leverage the standard technologies from large language modeling. And then in another area that we made a kind of adaptation of the standard tokenization scheme is in the case of BMFM-DNA model. So in this case, what we realized when we looked at existing models that are, you know, DNA language models where you basically map treat DNA sequences almost like a language where sequences of nucleotides are treated as words, and then use that in code biological function. What we saw that was that the preexisting models only used reference genome to train, and then they only represented reference genome, which is static. But then as we know, genomic variations, such as those cells captured in SNPs or single-nucleotide polymorphism, are what's driving a lot of biological functions in both in terms of your physiological traits, also in terms of your susceptibility to different diseases and your response to therapy. And so it's really important to be able to incorporate that in, you know, variation in the input sequence, right? So in order to do that, what we did was we pulled information from this database of SNPs, so it's called dbSNPs. We pulled 20 million different SNPs. And then from there, we're able to just identify at each position in that genome, whenever there's variation, we create a variation frequency matrix that captures the frequency of different genome variations. So there are five types of substitution, it's five types of insertion, and then one deletion. So that's altogether 11 types of different variation. And so we have this matrix that's by pulling from publicly available data to create this frequency matrix for every position where there is variation. And then we represent that matrix using a special character and then insert that special character back into the sequence. So now you have that sequence that's not disturbed, but then it represents, it has representation of genomic variation besides the static DNA sequence. And that achieves two goals. One is now you have the information on variation within this genome sequence, but also, you know, instead of prior approaches where SNPs are represented in isolation, you now have the representation of SNP in the context of that genome so that you can capture the relationships through their, you know, relative positions. And so that's another, you know, variation sort of, variation on the theme that we had to devise in order to be able to make use of these different multimodal data.
14:50 - Chris Wright
And this is all in pursuit of like the foundation model knowledge of biomolecular structures and how we respond to perturbations and--
15:04 - Jianying Hu
Yeah.
15:04 - Chris Wright
And therapies. So what if we shift gears and look at some of the applications?
15:10 - Jianying Hu
Yeah.
15:10 - Chris Wright
I know one of the challenges with drug discovery includes things like human trials, which can be really protracted time periods, for obvious safety and health reasons.
15:22 - Jianying Hu
Yeah.
15:23 - Chris Wright
Being able to accelerate those, not just from the discovery side, but then the efficacy and safety of the application of the therapy through simulations feels like
15:32 - Jianying Hu
Yeah.
15:32 - Chris Wright
A fantastic use for all of this knowledge that you're building from these foundation models.
15:37 - Jianying Hu
Yeah, yeah, absolutely, absolutely. So, yeah. So from that foundation, we need to then, you know, take additional data that's task-specific, that's domain-specific, and then use that to train through fine-tuning, you know, task-specific models to address real problems, right? In the biomedical research space. And, you're absolutely right, one of those, you know, ripe areas for AI to create an impact is exactly by, you know, connecting these in vitro experiments with in silico models and drive a tighter loop of, you know, iterative discovery process, right? So obviously we cannot do this on our own. We work with a lot of partners, different partners to, you know, identify these use cases with, you know, with real world discovery tasks and then use their specific data sets to train our models. And then in that process also really validate and enhance our methodology as well as the models. So one of, so we work with a lot of strategic partners, and one of our most important partner is Cleveland Clinic. So, you know, the Cleveland Clinic and IBM entered a 10-year partnership at the end of 2021 with the goal of really creating this discovery accelerator. Which is, you know, a joint center with the goal of, you know, leveraging the clinical and research capabilities from the Cleveland Clinic and IBM's, you know, global leadership on computing technologies to together drive, you know, faster discovery within the healthcare and life sciences space. So we have been working with many, many different research scientists in the context of this discovery accelerator, exactly for that purpose of, you know, identifying meaningful, you know, real world scientific discovery problems within the space and then to demonstrate the impact of these biomedical foundation models. You know, one specific example, for example, is we worked with Dr. Thad Stappenbeck in really creating a new, a research platform to be able to study the function, the progression, the function, as well as pathogenesis of human intestine, which is very important in, you know, arriving at therapies for conditions like IBD. So in this case, Dr. Stappenbeck's lab created this very novel experimental system that allows you to support a self-sustaining and two dimensional and long-lasting cultivation of cells from the human intestinal epithelial system. And then through a-single-cell transcriptomic analysis, they were able to identify different cell types that's differentiated. Both, you know, major lineages as well as minor populations. All very important in understanding how these different cells differentiate over time. And some of these differentiations will eventually lead to diseases, right? So it's, it holds a lot of promise, but in order to really fully, you know, utilize this novel system, we need to be able to establish the validity or the fidelity of results coming from this experimental in vitro system against the gold standard of biopsies that's taken from, you know, real patients. And that's a very, very difficult task. It's a challenge that the field has been facing for years. And what we were able to do is, we brought in our biomedical foundation model in collaboration, working with Dr. Stappenbeck's lab, we took the BMFM-RNA model, we fine-tuned it using publicly available data that's labeled data from sort of gold standard that's taken from patient biopsies, and then created this model to classify, identify cell types within specific for human intestine epithelial cells. And then once we have that fine-tuned model, we built this comparator kind of workflow to apply this classification to cells coming from different systems, including the in vitro experimental system, as well as the in vivo gold standard system, and then compare them. And by doing that, we were able to perform a really quantitative benchmark of the fidelity of results coming from the in vitro system against the gold standard in vivo system. And that really established sort of confidence in this new experimental system and then this new way of doing experiments by combining the in silico system with in vitro system and driving discovery in this space.
20:53 - Chris Wright
One of the things we didn't touch on is the ability to create with all of this foundational knowledge. And I see the applicability of fine-tuning--
21:04 - Jianying Hu
Yeah.
21:04 - Chris Wright
Something like very personalized healthcare. So it's not generalized therapeutics, but very personalized to given your own genetic makeup. And is that something that you feel is a useful outcome or are you taking it from a very different point of view?
21:25 - Jianying Hu
It is particularly in with the omics models that we have, that that really can drive a lot of studies on this notion of stratification. So very often what we call one disease is not a single disease. It's many, many different subphenotypes. Many, you know, different patients have very different manifestation and different response to therapy. So these, our omics foundation models would really help the understanding of what drives these different subphenotypes and how do you classify patients into these different subphenotypes and that has many different uses. One is to drive personalized care, as you mentioned, but it also can be used to drive more effective clinical trials because then you can use these biomarkers, if you will, to enrich trials so that you can enroll patients who the patients who belong to the right subphenotype, if you will, who are more likely to respond to the medication so that you are more likely to demonstrate the effect of the therapy. So it has use in that discovery space as well.
22:33 - Chris Wright
Well, why don't we look forward a little bit, you're working on this amazing research and great technologies. A lot of it is, not to reuse the word, but foundational.
22:44 - Jianying Hu
Yeah.
22:44 - Chris Wright
As we look forward, what are you most excited about the applications and the potential for healthcare and you know, AI ML coming together?
22:55 - Jianying Hu
Yeah, so I think one of the things that's really has been, and I think it all really continued to drive advancing this space, is really open science and open innovation, right? It's for many reasons. It's, you know, like I mentioned, you having, you know, open kind of innovation system is really, ecosystem is really the best way to ensure multidisciplinary and cross-domain participation in this innovation process. And as I mentioned before, it's really hugely important for deriving models that are really useful and meaningful for real world discovery use cases. And it's also really crucial in us being able to, you know, not just leverage the different talents from this diverse, you know, wider pool of participation, but also driving efforts of developing open and widely accepted and meaningful benchmarks. I think that's really another very, very important requirement for us to move forward is to have these benchmarks. I think having open and meaningful benchmark is one of the requirements. It's often very underrated, and I'll use that AlphaFold example again. So, you know, when it shared the Nobel Prize for chemistry last year, a lot of people were very quick to recognize that okay, they wouldn't have been able to achieve what they achieved without access to data coming from the protein data bank, which is publicly available. That has been created over many, many years, very painstakingly. And so that's one crucial thing for them to achieve that. But another very important factor that I think is not as widely recognized is it also depended to a very large degree, the existence of CASP, which is this open competition that has a very well organized set of benchmarks and yearly competition that's very, very widely participated by the whole community. And that allowed DeepMind to set a very meaningful goal for AlphaFold and to have a very focused and continuous effort to drive--
25:18 - Chris Wright
Yeah.
25:18 - Jianying Hu
You know, continuous improvement against these benchmarks until they eventually won it, right? So unfortunately, this kind of benchmarks is still very limited in the biomedical research space and even more so in clinical research space. So I think there needs to be a lot of effort in really promoting and establishing open science, open innovation efforts to create this kind of benchmarks to drive the field forward.
25:48 - Chris Wright
I, I love that you brought that up for two different things. The open science, the collaborative view of innovation and putting our collective minds together to solve a bigger problem. Obviously coming from an open source background, very well aligned with that viewpoint. And I absolutely appreciate the power of what we can do collectively versus individually. But also that undervalued view of benchmarks. I have to say, really wonderful learning from you, hearing what you're working on. I really appreciate your time. Thank you.
26:22 - Jianying Hu
Thank you.
26:23 - Chris Wright
What a fascinating look at the next frontier for AI with Jianying Hu today. We've seen AI grasp the different nuances of human language, but today's discussion shows how those architectural learnings are being translated to an equally, or possibly more complex domain: the language of biology itself. And I love the point Jianying made: it's not just about the AI model; it's about combining that power with deep expertise for tokenization of sequences and specific modalities, as well as across different scientific fields. This foundational, collaborative work doesn't always get the visibility that it should, but it's what truly makes discovery possible. It's an exciting glimpse into a future where we're not just asking AI what we know, but combining our collective human knowledge to empower it to discover what's next. Thanks for joining the conversation. I'm Chris Wright and I can't wait to see what we explore next on Technically Speaking.
About the show
Technically Speaking
What’s next for enterprise IT? No one has all the answers—But CTO Chris Wright knows the tech experts and industry leaders who are working on them.
