Scientists should beware using artificial intelligence in research as the technology may have been trained using faulty or biased datasets, a Nobel laureate has said.

While researchers have been quick to embrace the new technology, Avram Hershko, who won the Nobel Prize in Chemistry in 2004, told a scientific conference in Germany that he had concerns over the uncritical way in which some scientists accepted findings generated by large language models such as ChatGPT.

“We have to know what datasets it is using – it should be transparent,” Professor Hershko, who is based at Technion Israel Institute of Technology, told the annual Lindau Nobel Laureate Meeting, a gathering of 40 Nobel prizewinners and hundreds of young scientists.

The apparent certainty of results and recommendations generated by large language models trained on millions of scientific journals and books should also be addressed, said the Hungarian-Israeli biochemist. “It should say what the margin of certainty is…and this should be part of regulation,” he added.

̽����Ƶ

However, Michael Levitt, a University of Stanford structural biologist who won the chemistry Nobel in 2013, said scientists were often required to work with less-than-certain data and should be able to critically assess ChatGPT-generated data in a similar way.

“We expect data to be wrong all the time – science is very good at filtering these [bad results] out,” he said.

̽����Ƶ

Medical scientists often did not have full access to results behind discoveries, treatments or drugs but were still able to use less-than-complete datasets in a productive way, continued Professor Levitt.

“When the first Covid vaccines were designed, getting hold of the data was impossible – we often have to work with systems that are corrupted,” he said, adding: “We can use AI without having all the details.”

That said, Professor Levitt believed the most commercially successful AI models would be those that were transparent about the data they were reading because consumers preferred to understand more about their processes.

“That’s one reason why transparency will come…not because [these companies] want to be particularly open,” he said.

̽����Ƶ

The panel on AI’s likely impact on research also raised concerns over the “one-way” transfer of data flows from academia and scientific literature into the large language models owned by private companies, which help to make the AI models more powerful. In contrast, industry was not required to make its data open, limiting the ability of academia to develop comparable tools, the panel heard.

On this issue, Professor Hershko said that “information should be shared and AI should be open”.

“The only way to get around this problem is to make data open access so everyone can use it,” he said.

However, Aaron Ciechanover, who shared the Nobel Prize with Professor Hershko for their work on cell degradation, said this was unrealistic.

̽����Ƶ

“I would like data to be open but it will not be [possible],” he said, adding that drug companies and health insurance companies would fear that their data would be raided by their commercial rivals if it was made public.

“If you open up your data, other companies will make billions of dollars from it – not just from one thing, it would be endless,” said the Israeli biologist on the ways that the data would be exploited.

̽����Ƶ

Instead, companies who held data should be allowed to “make one type of money” in terms of using the information and “then the data is open”, he said.

jack.grove@timeshighereducation.com