Lost in the crush of recent news from this month’s SC22 event was the announcement that a research project involving Nvidia and several research institutions has made progress using large language models (LLMs) to predict variants of concern in SARS-CoV-2, the virus behind Covid-19.
In fact, the project was the winner of the Gordon Bell special prize, which essentially is the high-performance computing sector’s version of a Nobel Prize, during SC22. The effort involved more than two dozen academic and commercial researchers from Argonne National Laboratory, Nvidia, the University of Chicago and others.
According to an Nvidia blog post, the research team “trained an LLM to track genetic mutations in the virus. “While most LLMs applied to biology to date have been trained on datasets of small molecules or proteins, this project is one of the first models trained on raw nucleotide sequences — the smallest units of DNA and RNA,” the blog post stated.
“We hypothesized that moving from protein-level to gene-level data might help us build better models to understand COVID variants,” said Arvind Ramanathan, computational biologist at Argonne, who led the project. “By training our model to track the entire genome and all the changes that appear in its evolution, we can make better predictions about not just COVID, but any disease with enough genomic data.”
More details on how that was accomplished can be found in the blog post, but Nvidia said the research paper on the project indicated that the new model could be integrated with popular protein-structure-prediction models like AlphaFold and OpenFold, the latter of which is one of the pretrained language models included in the Nvidia BioNeMo LLM service for developers applying LLMs to digital biology and chemistry applications.
Nvidia’s efforts to support LLM training were a major topic of discussion at the company’s recent GTC Fall conference, where the company announced early access to two cloud-based LLMs, including BioNeMo.
The research team in this case developed its AI models on supercomputers powered by Nvidia A100 Tensor Core GPUs, including Argonne’s Polaris, the U.S. Department of Energy’s Perlmutter, and Nvidia’s in-house Selene system.