SONIFIED BALANCE BIOFEEDBACK CURRENT PROJECTS

 

Sonified biofeedback conveys biomechanics metrics via sound. We are developing systems with which people can interactively “tune” music biofeedback to tune their movement strategies. Recently, our lab has been awarded with an NSF CAREER award titled “Adaptive Biofeedback to Improve Balance during Everyday Mobility” to advance this mission. Additionally, we have support from K12HD073945. https://irek12.org/

There are two major objectives associated with this grant. One is to understand balance during turns of varied environmental contexts (I.) and the second is to use real-time sonified biofeedback to improve balance (II.). Jump to “II. Real-Time Sonified Balance Biofeedback”

I. Balance and momenta control during turns

Turns are ubiquitous in daily mobility and challenge balance, especially in fall-risk populations. In the first objective of the CAREER project, we are focused on understanding person-specific balance and momenta control strategies used during turns of varied environmental contexts across the lifespan. We have two papers published related to young adult balance and momenta regulation (see below), and we’re preparing two papers about how older adults regulate balance and momenta during turns- stay tuned!

Publications:

Tillman, M., Molino, J. & Zaferiou, A.M. Gait-phase specific transverse-plane momenta generation during pre-planned and late-cued 90 degree turns while walking. Sci Rep 13, 6846 (2023). https://doi.org/10.1038/s41598-023-33667-1

Tillman M, Molino J, Zaferiou AM. Frontal plane balance during pre-planned and late-cued 90 degree turns while walking. J Biomech. 2022 Aug;141:111206. doi: 10.1016/j.jbiomech.2022.111206. Epub 2022 Jun 20. PMID: 35772242.

 

First-person point of view in the two turn conditions and one “catch trial” where the cue to turn was not present, so the participant continued walking straight.

We are currently studying how balance is regulated during planned and late-cued turns. In the late-cued turns, participants know that there is a 50% chance that the monitor will display the grocery item of interest (green broccoli), as a cue to turn, or a “NO” symbol (red circle with a line through it), as a cue to continuing to walk straight.

The videos below are from the same exemplar participant during pre-planned and late-cued turns. Each participant either started walking with their left or right foot (e.g., “Start Left”) which led to different footfall strategies adopted when they reached the aisle.

During pre-planned turns, 10 young adult participants used 50% incidence of “step” or “spin” turn strategies that depended on the foot with which they initiated gait (in all but two trials). These strategies are summarized below:

  • A spin turn is marked by the foot on the side of the body in the desired direction of travel (interior to the turn; e.g., left foot during left turn) being placed closest to the intersection of the Center of Mass (weighted average location of mass, “COM”) approach and departure paths.

  • A step turn is marked by the foot on the opposite side of the body in the desired direction of travel (exterior to the turn; e.g., right foot during left turn) being placed closest to the intersection of the COM approach and departure paths.

During late-cued turns, about 70% of trials used a step turn strategy. More research is merited to understand if the step turn was decided due to the cueing time or to facilitate balance, as step turns are thought to be more balanced. Relatedly, we found that during these leftward late-cued turns, the COM was not as close to the edge of the left foot than during the pre-planned turns, during which participants COM was situated leftward of their left foot.

During the pre-planned turns, we can observe that the turn occurs over multiple footfalls. The overhead view demonstrates the curved horizontal center of mass trajectory with the body gradually reorienting to the desired facing direction. In contrast, the late-cued turns were performed with fewer footfalls and sharper turn radii. This pattern is consistent with having less time and space to execute the turn.

Our newest discovery relates to understanding how the body generates the requisite changes in horizontal plane angular and linear momentum. We are finding that regardless of the footfall strategy used, transverse-plane dynamics previously established during straight-line gait are used or leveraged during pre-planned and late-cued turns.

During straight-line gait, the body generates the most leftward angular momentum about a vertical axis immediately following left heel strike, during left double support. Likewise, during straight-line gait, the body generates the most leftward linear momentum during right foot single support. During pre-planned and late-cued leftward turns, we have found the same two strategies.

II. Real-Time Sonified Balance Biofeedback

We are currently evaluating how older adults interact with balance biofeedback systems with guidance from a physical therapist to improve their balance during walking and turning. Below, we include some example early prototype sound designs developed by Dr. Luke Dahl (UVA) and Matias Vilaplana Stark. We are

Publications

Tillman, M., Dahl, L., Knowlton, C., Zaferiou, A. (2020) Proceedings of the 7th International Conference on Movement and Computing. https://doi.org/10.1145/3401956.3404244

Dahl, L., Knowlton, C., Zaferiou, A. (2019) Proceedings of the 6th International Conference on Movement and Computing. doi 10.1145/3347122.3359600

DeSIGN PROTOTYPES:

audiovisual example of dance sonification:

This example sound design, by Dr. Luke Dahl, was used in the context of dance to explore the relationship between pelvis and feet, generally associated with whole body balance regulation. We currently use this design during STEAM outreach events.

 

We are working on more mobile ways to sonify balance metrics. Summer 2020 undergraduate Pinnacle Scholar, Paul Brellis, created this system to calculate the base of support and body’s center of mass using a Microsoft Kinect.

Video Details: On the left/below, you see the stick figure generated by Kinect. The middle you see the XYZ coordinates of the Body’s Center of Mass that uses assumptions about body segment parameter parameters. On the right, you can see the purple outline of the base of support along with the light blue dot as the projection of the body center of mass onto the ground.