Researchers from North Carolina State University (NC State), the University of North Carolina (UNC), and Arizona State University (ASU) have developed an intelligent system for tuning powered prosthetic knees, allowing patients to walk comfortably with the device in about ten minutes. The system is the first to rely solely on reinforcement learning to tune the robotic prosthesis, according to the research team. The new tuning system adjusts 12 different control parameters to address prosthesis dynamics, such as joint stiffness, throughout the gait cycle.
A paper about the development, “Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis,” was published January 16 in IEEE Transactions on Cybernetics.
“We begin by giving a patient a powered prosthetic knee with a randomly selected set of parameters,” says Helen Huang, PhD, co-author of the paper on the work and a professor in the Joint Department of Biomedical Engineering at NC State and UNC. “We then have the patient begin walking under controlled circumstances.
“Data on the device and the patient’s gait are collected via a suite of sensors in the device,” she says. “A computer model adapts parameters on the device and compares the patient’s gait to the profile of a normal walking gait in real time. The model can tell which parameter settings improve performance and which settings impair performance. Using reinforcement learning, the computational model can quickly identify the set of parameters that allows the patient to walk normally. Existing approaches, relying on trained clinicians, can take half a day.”
While the work is currently done in a controlled, clinical setting, one goal is to develop a wireless version of the system, which would allow users to continue fine-tuning the powered prosthesis parameters in real-world environments.
“This work was done for scenarios in which a patient is walking on a level surface, but in principle, we could also develop reinforcement learning controllers for situations such as ascending or descending stairs,” says Jennie Si, PhD, co-author of the paper and a professor of electrical, computer and energy engineering at ASU.
“I have worked on reinforcement learning from the dynamic system control perspective, which takes into account sensor noise, interference from the environment, and the demand of system safety and stability,” Si says. “We are thrilled to find out that our reinforcement learning control algorithm actually did learn to make the prosthetic device work as part of a human body in such an exciting applications setting.”
The researchers note that other questions will need to be addressed before the algorithm is available for widespread use.
“For example, the prosthesis tuning goal in this study is to meet normative knee motion in walking,” Huang says. “We did not consider other gait performance (such as gait symmetry) or the user’s preference. For another example, our tuning method can be used to fine-tune the device outside of the clinics and labs to make the system adaptive over time with the user’s need. However, we need to ensure the safety in real-world use since errors in control might lead to stumbling and falls. Additional testing is needed to show safety.”