The University of Central Florida invention comprises tactile-visual systems and methods for social interactions between isolated patients (for example, those with COVID-19) and remote visitors such as loved ones, family members, friends, or volunteers. A primary goal is to provide the isolated patient and the remote visitors with a visual interaction augmented by touch—a perception of being touched for the isolated patient and a perception of touching for the remote visitors. For example, a loved one might be able to virtually stroke the patient’s arm or head, or even squeeze the patient’s hand. A simple realization might include tactile transducer “strips” placed on the patient, with two-way video via touch-sensitive tablets, where touching the visual image of the strips on the tablet results in tactile sensations on the patient’s skin.

A more sophisticated realization could use the Physical-Virtual Patient Bed (PVPB), developed under NSF Award #1564065, to serve as a remote physical, visual, and tactile surrogate for the isolated patient. The remote visitors would be able to see, hear, and touch the PVPB. The isolated patient would see the remote visitors via video and feel their touch interactions via the tactile transducer strips on their arms and head (for example). These interactive video, voice, and touch interactions could provide additional comfort for the isolated patient and the remote visitors. Further embodiments and enhancements include 3D depth and viewing for visitors and patients, with possible 3D free space interaction. For example, visitors wearing augmented reality head-mounted displays could reach out and touch a virtual version of the patient, and the patient would feel tactile sensations. The systems and methods are usable in any conditions giving rise to isolation, including isolation due to geographical distance.

Benefit

Visual-motor synchrony established by the family and patient (seeing, touching, stroking, or grasping behaviors), coupled with the synchronized sensation of touch, will engender a sense of touching/being touched that brings comfort to the patient and loved ones

Market Application

Medical systems

Smart device manufacturers

Patient Examination Augmented Reality (PEAR) System

Abstract

The University of Central Florida invention is an approach that allows virtual overlays for manikins that enable realistic movement and a diversity of races/ethnicities and disorders while still providing physical contact. The innovation can be used to enhance any physical full-body patient simulator, manikin/mannequin, figure, doll, or other patient representation by using any video see-through head-mounted display (VST-HMD) or similar augmented reality (AR), virtual reality (VR), or mixed reality (MR) system to augment the appearance of the physical patient representation. It can also simulate physical movements, aspects, and behaviors that the physical patient representation cannot do on its own, while also affording the ability to touch the representation. For the healthcare domain, these enhancements offer several benefits, such as improved learning, training, mentoring, practice, and/or case planning. Other areas, including biology education, clothing retail, and any service or activity involving human representations may benefit from the invention.

Partnering Opportunity

The research team is seeking partners for licensing and/or research collaboration.

Stage of Development

Prototype available.

Benefit

Works with any physical patient representation, from simple dolls to complex patient simulators

Portable system

Affords more-realistic physical touching, grasping, and interactions, such as using a measuring tape to measure the circumference of a newborn’s chest