A DARPA-funded research team has demonstrated for the first time in a human a technology that allows an individual to experience the sensation of touch directly in the brain through a neural interface system connected to a robotic arm. By enabling two-way communication between brain and machine — outgoing signals for movement and inbound signals for sensation — the technology could ultimately support new ways for people to engage with each other and with the world.
The work was supported by DARPA’s Revolutionizing Prosthetics program, and performed by the University of Pittsburgh and the University of Pittsburgh Medical Center. The results were detailed in a study published online in the journal Science Translational Medicine, and the technology was among a number of advanced demonstrations presented to President Barack Obama at a White House innovation event in Pittsburgh in October.
“DARPA has previously demonstrated direct neural control of a robotic arm, and now we’ve completed the circuit, sending information from a robotic arm back to the brain,” said Justin Sanchez, director of DARPA’s Biological Technologies Office and the program manager for Revolutionizing Prosthetics. “This new capability fundamentally changes the relationship between humans and machines.”
The volunteer for the study, Nathan Copeland, has lived with quadriplegia from the upper chest down since a 2004 car accident that broke his neck and injured his spinal cord. Nearly 10 years following his accident, after agreeing to participate in clinical trials, Nathan underwent surgery to have four micro-electrode arrays — each about half the size of a shirt button — placed in his brain, two in the motor cortex and two in the sensory cortex regions that correspond to feeling in his fingers and palm. The researchers ran wires from those arrays to a robotic arm developed by the Applied Physics Laboratory (APL) at Johns Hopkins University. The APL arm contains sophisticated torque sensors that can detect when pressure is being applied to any of its fingers, and can convert those physical “sensations” into electrical signals that the wires carry back to the arrays in Nathan’s brain to provide precise patterns of stimulation to his sensory neurons.
In the very first set of tests, in which researchers gently touched each of the robotic fingers while Nathan was blindfolded, he was able to report with nearly 100 percent accuracy which finger was being touched. The feeling, he reported, was as if his own hand were being touched.
“At one point, instead of pressing one finger, the team decided to press two without telling him,” said Sanchez. “He responded in jest asking whether somebody was trying to play a trick on him. That is when we knew that the feelings he was perceiving through the robotic hand were near-natural.”
These latest results build on a series of DARPA achievements in directly interfacing the brain with a robotic arm. Earlier studies with volunteers Tim Hemmes and Jan Scheuermann demonstrated motor control of the APL arm using a brain-machine interface. “Based on DARPA’s success with those early tests, we asked, ‘Can we do the experiment in reverse and do for sensation what we did for the motor system?’” Sanchez said.
DARPA previewed its success with touch restoration in 2015 at “Wait, What? A Future Technology Forum,” an event that brought together thought leaders and expert scientists and engineers to generate new ideas and accelerate the development of novel capabilities. Full, peer-reviewed details of the research are described for the first time in the Science Translational Medicine article.
The interface system is one of two dozen technological breakthroughs that were on display at The White House Frontiers Conference, where Nathan and the lead researchers from Pitt talked about the technology, what it could mean for people living with spinal cord injury, and what new possibilities it could open for society.
Part of the President’s Brain Initiative, DARPA’s Revolutionizing Prosthetics program is funding research to refine stimulation patterns and incorporate new types of sensations beyond pressure to achieve the goals of delivering near-natural motor control and sensation to users of prosthetics. Improvements in these and related neurotechnologies could someday lead to near-seamless combinations of the cognitive functions of the human brain and the computing processes of machines.
Revolutionizing Prosthetics is not DARPA’s only program to pursue the restoration of a sense of touch to amputees. The Agency’s Hand Proprioception and Touch Interfaces (HAPTIX) program is pursuing an alternative approach, using the peripheral nervous system to communicate motor commands and sensory feedback between the brain and a prosthetic limb. The program plans to initiate take-home trials of a complete, FDA-approved HAPTIX prosthesis system by 2019.
A video of the demonstration of the brain-controlled robotic arm is available at www.youtube.com/watch?v=A4BR4Iqfy7w .
For more information, visit www.darpa.mil .
Transcript
00:00:03 I'm Justin Sanchez director of darpa's biological Technologies office today I'm really excited to share with you new results on how we are working to directly interface machines with the human brain a DARPA funded research team led by the University of Pittsburgh has demonstrated for the first time ever in humans the experience of the sensation of touch through a robotic prosthetic
00:00:26 arm connected directly to the brain the volunteer whose name is Nathan underwent a surgery to have two microelectrode arrays placed in his sensory cortex this is a region of the brain responsible for identifying tactile Sensations such as pressure those electrodes were then connected by wires to a robotic hand and arm fitted with tactile sensors in its
00:00:54 fingers index ring what you're seeing as the trials take place is that as the researcher applies light pressure to the robotic fingers those physical Sensations are converted into electrical signals that are fed directly back into Nathan's brain through this brain machine interface electrical signals are delivered as precise stimulation that
00:01:19 his brain interprets as though his own fingers are being touched index index ring despite being blindfolded Nathan can identify with nearly 100% accuracy pinky which fingers on the robotic hand are being touched 20 out of 20 what does this mean for the future of neurotechnology DARPA has previously shown that a brain interface can be used
00:01:48 to direct the movements of a robotic arm now with this new development of adding sensation by directly sending signals from the robotic hand back into the brain we have closed the loop between human and machine at DARPA we are always pushing the boundaries on what is possible we view neurotechnology as one of the next great Frontiers enabling new ways for humans to interact with each
00:02:12 other and with the world
