In 2005 Matt Nagle, here controlling the position of the light square on the computer, became the first person to control a robotic arm by his thoughts, using the BrainGate system from Cyberkinetics, Foxboro, Mass.
Turning thoughts into action
36
Medical Design • March 2007
Victoria Reitz
Senior Editor
I
Brain-computer interfaces take the thoughts of paralyzed people and turn them into commands by detecting electrical activity in the brain.
magine a machine that can sense what you think and act on your commands. Sound scary? Not so for people with paralyzed limbs or debilitating conditions such as Amyotrophic Lateral Sclerosis (Lou Gehrig’s disease). Machines like this could let them communicate and even move artificial limbs. Many diseases that paralyze people leave their brains unaffected. These people can think about moving or talking but can’t because they have problems in their spinal cord, nerves, muscles, or maybe they don’t have a limb.
www.medicaldesign.com
The electrical data recorded by the Neurotrophic Electrode from Neural Signals is exported to command a computer that speaks a phoneme for every electrical event or spike.
Brain-computer interfaces (BCIs) provide a connection. They record electrical activity in the brain and translate it into real commands such as moving a computer cursor or controlling an electric wheelchair. BCIs, already implanted in humans and animals, have potential to change lives.
Restoring speech
For the past 10 years Philip Kennedy has been implanting humans with electrodes, refining ways to record, preserve, and separate signals from the brain. Kennedy started Neural Signals, Atlanta, to work on brain-to-computer interfacing and is now aiming at restoring speech. Patients enrolled in the study, typically brain-stem stroke victims, are implanted with what’s called a Neurotrophic Electrode. It is shaped like a cone with a hollow tip and four gold wires inside. “Brain tissue grows into the tip and we record electrical activity across the Teflon-insulated wires,” says Kennedy. The implant goes inside the brain tissue and is wirelessly powered by a power-induction system which lies just under the scalp. The system works like this: “We have the patient say a phoneme in his head,” says Kennedy. Phonemes are the smallest unit of sound in a language, for example, the ‘b’ in book, or ‘th’ in that. “There’s a different firing pattern for each phoneme. In one patient, the computer can recognize 32 of the 39 English phonemes.” “In the training, we asked the subject to listen to a phoneme produced by a computer. That person would have to say the phoneme in their head. If the computer interpreted it correctly, the computer rewww.medicaldesign.com
peats the phoneme. Sometimes we hit 80% accuracy, sometimes 50% or less,” Kennedy notes. And this is where the engineering challenges come in. For example, one patient in the study could produce some short phonemes like ma, da, and I,O, U, which is a big deal to researchers, but not much use to the patient. “The real challenge is to put the phonemes together as words,” Kennedy says. “Because phonemes can be produced as fast as twenty in a second, there’s a lot of data to sort out.” Kennedy and his team of mostly engineers are also working to get the signals to fire under the patient’s control better than they are right now. “We’re constantly improving the electronics,” he adds. They’re also working on the power-induction system, getting more signals, and improving the electrode. Kennedy’s goal in the next year is to have the patient generate a hundred short words. “We want to resynthesize the phonemes to produce some intelligible language. It will be five to ten years to get the system producing conversational speech.” Kennedy envisions this technology moving beyond medical applications. “There’s potential for enhancing healthy humans by improving memory storage and calculating abilities — almost as in the movie ‘The Matrix,’ where the main character uploads knowledge through a socket in the back of his head. I don’t know if we’ll be downloading exactly like the movie, but certainly this research could lead to something that today is unthinkable.”
March 2007 • Medical Design
The Neurotrophic Electrode from Neural Signals (laid across the penny) is implanted in brain tissue. Detected signals are sent to a transmitter which wirelessly sends the command to a computer.
37
Controlling movement
The five-year goal for mands. “The sensor detects impulses from another company is to have a few dozen to 150 cells,” says John a tetraplegic hold a spoonful Donoghue, founder and Chief Scientific of food and bring it up to his Officer of Cyberkinetics. One hundred mouth. BrainGate is a brain-computer in- gold wires extend from the array to a terface that also senses signals from the pedestal which extends through the scalp. brain and transforms them into action. The pedestal is connected by an external But Cyberkinetics Neurotechnology cable to what are typically called “assisSystems Inc., Foxboro, Mass., is using its tive technologies,” or devices such as a device to generate motor commands. computer or robotic arm. Its system is comprised of an implanted Cyberkinetics is running two studies, sensor, a decoder or translator, and a hu- one for people paralyzed from spinal cord man interface. The sensor is a 4 x 4-mm injuries or brain-stem strokes and one for silicon array with people with ALS, or 100 hair-like elecLou Gehrig’s disease. trodes. It is im- Watch BCIs in action “We have people iniplanted onto the sur- There are many online videos showing BCIs tially look at a comface of the brain in use. Find links to them by going to Vicki puter cursor that where motor com- Reitz’s blog at forums.medicaldesign.com we’re moving on the mands are generscreen and tell them and clicking on the post called “Watch BCIs ated, specifically to imagine they are arm motor com- in action.” moving it,” says
Medical Design • March 2007 www.medicaldesign.com
The BrainGate from Cyberkinetics detects electrical activity in the brain and uses the signals to command a computer outfitted with assistive technology. (Inset) Cyberkinetics sees its BrainGate system used as a way for paralyzed people to check email, switch a TV channel, control a wheelchair, and one day restore limb function.
38
Monkeys learn to control a cursor
Researchers at Duke University showed that monkeys can control a robot arm with just thoughts and visual feedback. Miguel Nicolelis, codirector of the Center for Neuroengineering, Duke University Medical Center, and his colleagues implanted an array of microelectrodes into the frontal and parietal lobes of the brains of two female rhesus macaque monkeys. The researchers implanted 96 electrodes in one animal and 320 in the other, and recorded and analyzed output signals in the initial experiments from the monkeys’ brains. They were taught to grasp a joystick with a specified force and use it to position a cursor over a target on a video screen. After initial training, researchers complicated the activity by letting the animals control a robotic arm. The arm mimicked the movement of the monkeys’ arms. The scientists then removed the joystick. The monkeys continued to move their arms in mid-air as if working an imaginary joystick to manipulate and “grab” the cursor, thus controlling the robot arm. “The most amazing result was that after only a few days of playing with the robot this way, the monkey suddenly realized she didn’t need to move her arm at all,” says Nicolelis. “Her arm muscles went completely quiet, she kept the arm at her side and controlled the robot arm using only her brain and visual feedback. Our analysis of brain signals showed that the animal learned to assimilate the robot arm into her brain as if it was her own arm.” “We know that the neurons from which we were recording could encode different kinds of information,” says Nicolelis. “But the surprise is that the animal learned to time the neuron activity to control different parameters. For example, after using a group of neurons to move the robot to a certain point, these same cells would produce the force output that the animal needed to hold an object. None of us had encountered an ability like that.”
Donoghue. “As they do that their brain cells change activity patterns. We map the relationship between that pattern of activity and the motion of the cursor. With a few quick practice blocks we build up a relationship that’s quite reliable. And we map the firing pattern onto the cursor motion.” The results are promising. Patients can control a simple computer interface to open e-mail, change the TV channel, or turn lights on and off. “We’ve demonstrated they can open and close a robotic hand, and control a robotic arm. One patient could operate an electric wheelchair. She wasn’t sitting in the chair but she was directing it around the room. We’re just not yet to the point where these are everyday operations,” he adds. The current device has a pedestal and is connected through the skin, so the patient is tethered to a computer through a cable. “We’re working on a fully-implantable wireless system. This will let the patient be mobile, and get rid of the percutaneous connector that passes through the skin,” he adds. But that’s going to take some time to develop. “We wanted to be sure the initial indications are that the concept would work.” Donoghue says the BrainGate could one day give patients as many functions as a normal hand. “We’re working on taking signals from patients, particularly spinal cord injuries, and returning them through the body to a stimulator hooked up to
www.medicaldesign.com
muscles,” he says. In other words, researchers could fully repair someone.
Less-invasive BCIs
Still other researchers are working on less-invasive BCIs. Jonathan Wolpaw is the head of the Wadsworth Brain-Computer Interface project at the Wadsworth Center, New York State Department of Health in Albany, N.Y. Their version of a BCI includes a Make contact laptop computer, Center for Neuroengineering, a portable ampli- Duke University fier, and a breathwww.duke.edu/~ch/Neuroeng/Neuro.htm able cap fitted with eight elec- Cyberkinetics www.cyberkinetics.com trodes. The cap Neural Signals www.neuralsignals.com records electrical Wadsworth Center www.bciresearch.org activity through the scalp, similar to an EEG, and translates it into commands. A caregiver must attach the electrodes to the patient every time the cap is worn. One patient has used the system for over a year, along with the computer’s software to compose e-mails and letters. It works this way: A screen shows a matrix of letters and numbers. The subject concentrates on the character he wants while the lines and columns flash. When he gets the correct one the process begins again. He is able to type at about two to four words per minute. ■
March 2007 • Medical Design
39