A recent study published in the journal Natural Neuroscience has shown the results of a new brainwave-reading machine. The computer uses a new algorithm that’s designed to recognize brainwave patterns and translate them into real-time sentences. In other words, the computer is translating what a person is saying not by listening to them but by monitoring their brainwaves.
The error rate of this attempt was just 3% which is a significant improvement over previous “brain-machine interfaces” attempts. Earlier algorithms were only able to decode fragments of a person’s speech and a very small percentage of the words were in coherent phrases.
Still, this isn’t brand new research as Brain-Computer interfaces and convoluted neural networks have been experimented with for quite a while. The authors of this recent study are building on quite a lot of previous work and this is just another step toward the goal of decoding brainwaves and patterns.
This recent success was accomplished by scientists from the University of California, San Francisco (UCSF) and included 4 volunteers who had to read sentences off scripts. While they were reading, their brain activity was recorded via electrodes and fed into a computing system that used the new algorithm to create a representation of the brain patterns.
How does the algorithm work?
The algorithm is designed to do several things simultaneously. First, it looks for repeated features of speech such as commands to parts of the mouth, consonants, vowels, etc. Together with that, the algorithm also tries to decode these representations word-by-word and form sentences out of them. That second part of the algorithm seems to be what’s giving it most of its recent success in not just hitting or missing certain words but successfully forming phrases and sentences.
It’s not perfect yet, however, and the researchers readily admit that. For one, the speeches they used in the study were intentionally limited to only 30-50 sentences.
“Although we should like the decoder to learn and exploit the regularities of the language, it remains to show how many data would be required to expand from our tiny languages to a more general form of English,” the researchers shared in the Nature Neuroscience paper.
The scientists are hopeful that the algorithm will soon be able to understand and decode sentences that it’s never encountered in training. They base that on the fact that they added extra sentences in the study that they hadn’t used in their initial testing and the algorithm managed to decode them as well.
Another encouraging factor was that the machine got better at decoding sentences as they switched the volunteers, meaning that the technique used by the software is transferable between people.