Professors team up to design a better prosthetic arm

If computer scientists and engineers join forces, can they build a better prosthetic arm, one that could even, say, play the piano? A pair of San Francisco State University researchers hopes to find out.

Kazunori Okada, an associate professor of computer science, and Xiaorong Zhang, an assistant professor of engineering, have embarked on a project to develop a prosthetic arm that better understands and interprets the complex electromyogram (EMG) signals necessary for elaborate arm, wrist, hand and finger movements. EMG is a measure of electrical voltage generated from muscle contractions and contains information about the neural signals sent from the spinal cord to control the muscles.

"An arm is a basic tool of a human being, and losing it is a life-changing experience," Okada said. "Coming up with a good replacement for a lost arm is difficult. There are still muscles and nerves at the end of the arm, but how they coordinate together is very complex and reading their signals is not an easy thing."

The project is the first to receive funding from the Ken Fong Translational Research Fund, which was established through a $5 million donation from SF State alum Ken Fong to support interdisciplinary research projects. The fund has backed Okada and Zhang’s venture with a $20,000 grant.

Today's EMG control technology, which is based on "single-channel" EMG recordings on multiple muscles, can only recognize simple static motions such as hand open or closed due to the lack of meaningful neuromuscular information that can be captured. But arms and hands are highly complex, the researchers say — multiple muscles must fire at varied intensities and often in a set sequence to perform basic tasks.

"Existing design only allows a few simple motions, like opening or closing a hand," Zhang said. "These are static motions. But if you were to grab a glass of water and drink it, that's a sequence of motions that is continuous
and dynamic."

To solve the challenge of allowing more complex movements, the project will build on a concept by Zhang that envisions a grid of signal readers capable of capturing richer neural information across both space and time. She and Okada will begin their research by capturing EMG signals from live subjects using electrode grids and analyzing how effective they are in representing the proportional and dynamic muscle activities of hand gestures. Zhang will also use her expertise in embedded computer system design to develop a high-performance, real-time computing system to address the computational challenges of applying grid sensing to real-time
prosthetic control.

Okada, a music lover, uses the metaphor of a recording studio to explain the concept. Highly trained, professional engineers are brought in to precisely place dozens of microphones and accurately capture the sound. But what if someone were able to develop a single structure with multiple microphones already in the proper locations and backed by a computer that could interpret and isolate the sounds?

That is where Okada's expertise in machine learning comes in. As the signals are collected, a computer program will be built to not just read and interpret them but also test out solutions, learn from mistakes and adapt.

The ultimate goal is a prosthetic arm that non-professionals or the wearers themselves can place onto the body and, without any noticeable delay, use it to control all of the complex motions of a regular hand, from eating and drinking to playing the piano. But Okada and Zhang acknowledge that such a finished product is likely a long way away. For now, they are focused on collecting data and building computer software that analyzes that data and points them toward a solution. The process will involve working with undergraduate and graduate students, something both researchers say reinforces the value of working across departmental lines.

Such interdisciplinary thinking is often missing in students' crowded study schedules, but is essential in their learning processes and for their future careers, Okada said.

"In the real world, to solve challenges, you need to break pre-existing dogmas and come up with new solutions, so this kind of collaboration is great stimulation, not only to students but also to faculty," he said. "That's why I'm very excited."

To learn more about SF State's Department of Computer Science, visit http://cs.sfsu.edu/. For more about the School of Engineering, visit http://engineering.sfsu.edu.