TECH

Will AI Graduate from Tool to Lab Member? A Q&A with John Tsang

John Tsang, PhD, and his Yale lab members Jacob Kim and Ao Huang recently posed the question of whether AI immunologists are ready for prime time. Specifically, they were referring to large language models (LLMs) and whether they are as creative as humans in developing new hypotheses and methods for research in the field of immunology.

The answer, based on current studies, is no, not yet, says Tsang, Anthony N. Brady Professor of Immunobiology and professor of biomedical engineering at Yale School of Medicine. While there are functions where AI can be effective, such as summarizing relevant literature, LLMs like ChatGPT aren’t able to consistently generate truly original hypotheses, scientific ideas, or experimental approaches.

But, Tsang adds, researchers are exploring new ways of engaging LLMs that could overcome these current hurdles.

“I think there is a future—probably not too distant—where AI will be able to do such things,” says Tsang, founding director of the Yale Center for Systems and Engineering Immunology and an investigator with and the Yale lead for Biohub New York. “I see AI one day being a sort of team member, contributing ideas alongside scientists.”

We spoke to Tsang about the current state of what he calls AI immunologists and where he sees opportunities going forward.

John Tsang, PhD: An AI immunologist isn’t yet a robot that goes into the lab and does experiments. It’s an AI system that can think through concepts, propose hypotheses, suggest experiments, and interpret the data that come out of those experiments. That’s the kind of AI immunologist we’re talking about.

What do you mean when you refer to AI immunologists?

John Tsang, PhD: An AI immunologist isn’t yet a robot that goes into the lab and does experiments. It’s an AI system that can think through concepts, propose hypotheses, suggest experiments, and interpret the data that come out of those experiments. That’s the kind of AI immunologist we’re talking about.

How is AI capable in that regard now? What can it do well?

Tsang: What it can do really well is research the literature. For example, if you ask, “What's known about this molecule? What can that molecule do to this kind of cell?” AI can deliver excellent information. Basically, if you talk to any of these AI chatbots, things like Claude, Gemini, and ChatGPT, they're doing very well in this regard these days.

And where does AI still have room for improvement?

Tsang: AI is still not super effective at coming up with things that may be novel in the sense that AI may not be completely taking into account, for example, information from another field. If you don't prompt the AI and say, “Hey, have you thought about this particular knowledge in biochemistry or this particular thing over in aging? Do you think there's a connection between that and this principle?” it doesn’t do that on its own.

You note there are exciting opportunities when it comes to multi-agent approaches. Can you describe what these approaches are?

Tsang: When using AI in a classical way, you have one agent, and you have an interactive discussion with it. With a multi-agent approach, as a few research groups have now described, you ask different independent bots to take different roles. One bot could be a computational biologist, another one could be a biochemist, and yet another one could be someone who knows a lot about antibodies, for example. And then there's also a human who's involved in the team that can give the bots a high-level task and then let them talk to each other.

What is very interesting is that in examples of this type of approach so far, the majority of the interactions happen among the AI bots. The human doesn't necessarily go in and tell them what do. And researchers have found that the bots are capable of coming up with a work plan and executing it. Even when the human, while part of the team, is not doing the major coordination.

So that is quite interesting, suggesting that by having interactions among these bots—and I think the key is that they take on different roles—they are able to start to generate something that's quite emergent, not just straight answering. They can engage a multi-step plan and make decisions based on the data. That is, I think, quite intriguing.

One striking example is “The Virtual Lab,” a system developed by Biohub colleagues in which an AI principal investigator led a team of AI specialists—an immunologist, a computational biologist, and a machine learning expert—to design nanobodies (small antibody-like molecules) against a new SARS-CoV-2 variant. The team produced 92 candidate designs, and two were experimentally confirmed to bind the new variant. Notably, more than 98% of the words exchanged during the project came from the AI agents themselves; the human researcher set the goal and stayed in the loop, but the bulk of the back-and-forth largely happened among AI agents.

Would this type of approach help overcome some of the current limitations of AI?

Tsang: Yes, I think so. And I think it also overcomes what's called a context window problem. So each bot has a certain memory, and when you have a team like that, the total volume of memory becomes quite large. At the same time, the bots can start to write out answers to external files, which can get pulled back in again, extending memory further, in a way.

Another interesting thing that's emerging is what's called “world models.” So these are agents that have a much longer memory span, so they're not just working within this context window. They have external storage, and they try to provide a model of the world that the particular agent is in. I think that's also going to help.

Where do you think is the greatest potential when it comes to AI and immunology?

Tsang: I think the answer lies on two levels. First, I think it's going to be helpful in proposing and hypothesizing things that we haven't thought of yet. However, that will be based on existing data, and we still have limited data on lots of things in immunology.

For example, one project that I co-lead called the Human Immunome Project aims to capture human immunology variation across different people and across different populations. Now that kind of data simply doesn't exist yet. If you ask a chat bot or even multi-agents and say, "I know this vaccine works in the U.S. with this level of efficacy, what happens if I bring it to Tanzania?" it would have no idea because it simply doesn't have enough information and knowledge yet.

So that's where a different kind of AI, models like quantitative models of the human immune system, come in. That's another area that we are actively working on, how to develop that kind of model. Of course, once such models become available and useful, then AI agents can use them to query and answer the questions we ask.

Additionally, AI agents, I think, could form a team with us to work together on that problem, where we need to design experiments. We need to think about, given a certain number of resources and budget, what's the best first experiment, for example, what's the best first data collection exercise? Those are the kind of things that we are thinking about a lot.

How do these AI technologies intersect with education?

Tsang: I think we need to put these into education fairly quickly so students can understand how to use them and feel comfortable doing so. Plus, there are creative aspects, like the multi-agents and how to put them together.

I personally use these tools, especially when I’m starting something new. I want to get a sense of what’s known and also to potentially make novel connections. And my lab has discussed this sort of AI team approach.

Sometimes I think there's a bit of a worry about using AI to cheat, for example. That's what some educators may be thinking. But I think we have an opportunity to promote positive and productive use. And so the question is how to incorporate this technology in the most positive, effective way possible.

by: Mallory Locklear, PhD--Managing Editor—Science, Research, and Education