We are seeing lots of developments involving the application of virtual reality (VR), extended reality, and augmented reality in healthcare and medicine. VR healthcare companies are applying these technologies to create virtual environments where patients and caregivers can participate in highly realistic and interactive experiences designed to simulate various healthcare scenarios and needs. Key application areas include education/training, treatment, and physical therapy/rehabilitation.
This Advisor series examines the use of VR in healthcare and medicine. Part I covered how companies are using VR technologies to develop applications for healthcare educational and training purposes. Part II focused on the use of VR for treatment, therapy, and rehabilitation. Here in Part III, we examine developments involving digital humans for healthcare and medicine.
Digital Human Technology in Healthcare
Digital humans are digital representations of a human being; however, they do not have to be based on an actual person. Their role is similar to intelligent virtual agents and other self-service applications. In fact, digital humans often function as a friendly face in the form of a highly photorealistic and interactive 3D image designed to operate as a front end to such applications.
Digital humans make extensive use of artificial intelligence (AI) — including natural language processing (NLP), natural language generation, and speech recognition/generation — to function in a human-like manner, including fielding and answering questions on particular topics in real time.
Digital humans also employ 3D interactive modeling and animation to realistically simulate human movements involving the eyes, mouth, and other facial expressions made by people when talking, listening to, or interacting with others. Because they are designed to offer a more “sympathetic ear” to end users, they typically employ emotion recognition and sentiment analysis to interpret the emotions, behavior, and speech of the customer, client, or patient with whom they are interacting. You might think of digital humans as an intelligent avatar designed for highly interactive, immersive, and engaging virtual applications.
Companies are developing digital humans for various healthcare and wellness roles, including: training and education, emotional support coaching, symptom gathering, clinician decision support, and remote patient monitoring. Current applications range from digital humans designed to provide advice on various diseases and conditions to correct prescription drug usage.
Next, we look at some representative examples of digital human healthcare applications currently available or in various stages of development.
COVID-19 Digital Human Public Health Advisor
In April 2020, digital human platform provider UneeQ developed “Sophie,” a digital human designed to offer advice to the public about COVID-19. At that time, the virus was poorly understood. Lots of misinformation about symptoms, contagiousness, and treatment was circulating. Consequently, UneeQ decided to develop Sophie to answer some of the common questions about the virus.
UneeQ developers used IBM’s Watson Discovery cognitive development tool and Google speech-to-text engine to implement Sophie’s NLP capabilities, which were integrated with the US Centers for Disease Control and World Health Organization’s databases and other sources. This allowed Sophie to autonomously access the latest COVID information when responding to questions fielded by the public. (You can watch an overview of Sophie here.)
MayaMD: A Digital Human Health Assistant
MayaMD offers a personal health assistant that patients can use on their smartphones. MayaMD’s assistant is based on the company’s clinical AI technology. Its models were trained on a library of evidence-based medical knowledge, allowing the app to interpret complex clinical data in response to patient symptoms.
Interacting with MayaMD mimics a conversation a patient might have with their doctor about symptoms they are experiencing. Upon entering their symptoms into the app, the user is led through a Q&A session to gather additional information. MayaMD then offers possible diagnoses sorted by probability and severity as well as the steps necessary to further refine the diagnosis (e.g., recommended lab work, X-rays). It also recommends the type of clinician and care facility the patient should contact and lists local providers. Finally, MayaMD generates a clinical note the user can forward to the care provider they intend to contact. MayaMD also offers a clinical decision support tool for clinicians (“Maya Pro”). MayaMD and Maya Pro are available on the iTunes and Google Play app stores.
Digital Human Pharmacist
Tech start-up Pharmi partnered with Deloitte and Sint Maartenskliniek hospital, in the Netherlands, to develop a digital pharmacist, which will be used to advise rheumatism patients with the correct use of the drug methotrexate. The digital human is designed to answer common questions about the drug (e.g., when to take it, possible side effects). Sint Maartenskliniek hospital is testing the application with rheumatism patients in its hospital network.
The digital pharmacist is built on top of Pharmi’s existing digital medication advisory system (MedWise), a mobile app that provides 24/7 information and digital care for many medications on the Dutch market. Because of this, it is the intention that the digital human pharmacist will eventually be able to provide guidance on other medicines used by the hospital.
Conclusion
The use of digital humans in healthcare and medicine is really just beginning to get underway, with many of the applications still in development or testing stages. That said, we should expect the use of digital humans in healthcare to increase over the next few years as the technology advances. In addition to use for training and educational purposes, the human-like functionality of digital humans makes them well-suited for healthcare advisory applications where patients need to feel that they are appreciated and that their concerns are being heard and adequately addressed (which is in stark contrast to traditional self-service Web applications and sterile chatbot interactions). Healthcare providers will want to use digital humans to contend with shortages of skilled personnel and to be able to provide a friendlier (i.e., better) customer experience.
Finally, I’d like to get your opinion on the application of VR in healthcare and medicine as well as its use in other industries. As always, your comments will be held in strict confidence. You can email me at experts@cutter.com or call +1 510 356 7299 with your comments.
