AI systems' signals found to mirror how brain interprets speech

New Delhi: Providing possible insights into how artificial intelligence (AI) systems operate, a new research showed that they can process signals in a way strikingly similar to how the brain interprets speech.

Researchers from the University of California, Berkeley, US, measured the brain waves as participants listened to a single syllable - "bah". They then compared that brain activity to the signals produced by an AI system trained to learn English.

A side-by-side comparison graph of the two signals was found to show a striking similarity. The researchers said that this was raw data and that the data had not been tweaked.

"Understanding how different architectures are similar or different from humans is important," said Gasper Begus, assistant professor of linguistics at UC Berkeley and lead author on the study published recently in the journal Scientific Reports.

That is because, he said, understanding how those signals compare to the brain activity of human beings is an important benchmark in the race to build increasingly powerful systems.

For example, Begus said, having that understanding could help put guardrails on increasingly powerful AI models. It could also improve our understanding of how errors and bias are baked into the learning processes.

To do so, Begus turned to his training in linguistics.

He said that the sound of spoken words enters our ears and gets converted into electrical signals, which then travel through the brainstem and to the outer parts of our brain.

Using electrodes, researchers traced that path in response to 3,000 repetitions of a single sound and found that the brain waves for speech closely followed the actual sounds of language.

The researchers transmitted the same recording of the "bah" sound through an unsupervised neural network - an AI system - that could interpret sound. They then measured the coinciding waves and documented them as they occurred.

Begus said he and his colleagues are collaborating with other researchers using brain imaging techniques to measure how these signals might compare. They're also studying how other languages, like Mandarin, are decoded in the brain differently and what that might indicate about knowledge.

Many models are trained on visual cues, like colours or written text — both of which have thousands of variations at the granular level. Language, however, opens the door for a more solid understanding, Begus said.

The English language, for example, has just a few dozen sounds.

"If you want to understand these models, you have to start with simple things. And speech is way easier to understand," Begus said.

In cognitive science, the researchers said, one of the primary goals is to build mathematical models that resemble humans as closely as possible. The newly documented similarities in brain waves and AI waves are a benchmark on how close researchers are to meeting that goal, they said.