ACCAD: Where art and tech meet to help people
This research center and Nationwide Children’s Hospital are working together to create virtual reality training that will help staff calm patients.
John Luna, interactive digital media developer at Nationwide Children’s Hospital, and Alice Grishchenko ’17 MFA work in ACCAD’s Motion Lab. Luna is a project leader and Grishchenko is contributing to both visual design and interactive programming. (Photo by Jodi Miller)
As a trained dancer and veteran of Ohio State’s Advanced Computing Center for the Arts and Design, John Luna ’17 MFA knows how pairing body movement, technology and imagination can create meaningful ways to help people.
“There’s an improvisational, exploratory, body-centered way of understanding things that makes for a better understanding of each other,” says Luna, who now works in the research division at Nationwide Children’s Hospital in Columbus. “I make interactive media tools and apps for kids, their families and health care providers. The work has spanned from interventions that help with pain management to educational tools or exposure therapies.”
His new project is leading a partnership between Nationwide Children’s and ACCAD, where he interned during his years as an Ohio State grad student in dance. They’ll use virtual reality simulation to train hospital staff on how to de-escalate patients, both pediatric and adult, who are in crisis, a project funded by the Ohio Bureau of Workers’ Compensation.
The approach is important to the children’s hospital because expanding innovative behavioral health care models plays a big part in its broader strategic plan.
“Improving outcomes of behavioral health starts with how you treat patients, what biases are present,” Luna says. “We want to focus on these high-risk, low-frequency events where the provider has to know how to calm themselves and the patient to control the environment.”
The proof-of-concept phase was completed in 2022. For this, trainees used a VR headset to watch video clips showing scenarios in which a patient’s behavior escalated. They would make choices — like in a Choose Your Own Adventure book — that determined which scenes they saw next.
After a year of studying how the staff responded to the video, Luna says, the research team proved the approach worked. ACCAD will be integral in completing the next stage of the training, which will be more immersive.
“The big difference here is that it typically takes about five people to train one person with in-person role-playing,” he says. “But with this experience, one person can train 10 people simultaneously.”
The project marks more than 10 years of ACCAD collaborations with hospitals and health care organizations, says Vita Berezina-Blackburn ’03 MFA, a co-lead for the children’s hospital project on Ohio State’s side and senior creative technologist with ACCAD.
The multidisciplinary research center, housed in the College of Arts and Sciences, includes specialized labs fueled by emerging technologies, computer animation, design and art. While it offers classes to students of many disciplines, its other major focus is drawing in students and faculty from across Ohio State to research how tech can improve people’s lives.
“The thing that’s great about working with a children’s hospital is, some of the elements of creativity that you would use in a Pixar movie, for example, have a lot of application to that audience in that space,” says Jeremy Patterson ’01, a graphics researcher and instructor with ACCAD for more than 15 years. He’s researching how people interact with augmented reality for the new project.
Grishchenko and Vita Berezina-Blackburn, senior creative technologist with ACCAD, adjust the VR headset controls while working in the Motion Lab, in Sullivant Hall. (Photo by Jodi Miller)
Moving beyond the proof of concept, that project’s second phase started this summer. For it, ACCAD is working with Nationwide Children’s to create virtual avatars and hospital rooms where employees can role-play scenarios with dialogue and movement.
ACCAD’s Motion Lab plays a big part in this step. The 2,500-square-foot room is equipped with a dozen state-of-the-art motion-capture cameras, multiple VR systems and a specialized facial-capture camera rig.
Berezina-Blackburn leads the team that records and integrates human motion and expressions, dialogue and narration using the lab’s motion capture system. That will provide the movement for the virtual characters who will appear in the training simulation. It’s the same technology used for feature films and video games.
Patterson will research methods to overlay the recorded motion with virtual characters in the augmented reality environment (a step toward better interactive spatial computing, such as afforded by the upcoming Apple Vision Pro headset). He says tapping this high-level technology for human interaction, and not just entertainment, is a hallmark of what puts ACCAD at the forefront of technology innovation.
And because ACCAD maximizes every learning opportunity, during this project Assistant Professor of Design Shadrick Addy will study training assessment, and Mirkamil Majid, a PhD student in computer science engineering, will focus on how AI-based algorithms can make avatar movement look and feel more realistic. “We want to explore which algorithms are preferable and what are the bottlenecks that affect the user experience,” Berezina-Blackburn says.
Students will gain valuable experience helping with each step, just as Luna did in his years at Ohio State.
“Before coming to ACCAD, I was a professional dancer and video artist. I always wanted to delve more into interactive digital works, and grad school provided that opportunity,” he says. “A lot of the work that I ended up doing at ACCAD was based on the idea that the body is this expressive vehicle and its movement has meaning.”
x
More ACCAD projects
Eyes of Mariam VR
This experience (above) shows how it feels to walk in the shoes of a girl seeking education in a fictional warring African country. The project helps people understand the challenges for kids who have experienced trauma.
Immersive Learning for Intel
Intel is building two semiconductor factories east of Columbus, expected to open in 2025, with up to six more possible. As part of a grant from Intel, ACCAD and the College of Engineering are exploring the use of virtual reality for immersive training to assist in building a diverse semiconductor workforce.
Japanese Butoh
ACCAD and Assistant Professor of Dance Tanya Calamoneri captured Minako Seki’s Butoh dancing and vocalizations, which will lead to learning tools for dancers.
Forensic reconstructions
If human remains can’t be ID’d, publicizing what the person might have looked like can help restore their name and provide closure to their family. With the Ohio Bureau of Criminal Investigation and Ohio State’s Dean Hensley, ACCAD researchers and students invented a fast process to create virtual facial reconstructions that can be shared with media and other sources, increasing the chance of solving missing persons cases across Ohio.
