A Study of Converging Technologies towards the Development of AR/VR-based u-Healthcare Systems

2.1 Wireless Sensor Networks (WSNs) in Healthcare

Wireless sensor network(WSN) refers to the utilization of spatially distributed autonomous and tiny devices capable of monitoring physiological and environmental conditions. In healthcare, these physiological and environmental conditions include blood pressure, temperature, pulse rates, glucose levels, and other bodily attributes[7][8]. The gathered healthcare information by the sensors were relayed to a gateway through wireless technologies such as RFID, ZigBee, or Bluetooth. Gateways will then forward these healthcare information into the processing nodes (e.g., smartphone, tablet, laptop, desktop computer, or more sophisticated server machine) by using wireless protocols that include available standards such as 2.4 GHz radios based on either IEEE 802.15.4 or IEEE 802.11(Wi-Fi) standards.

The utilization of WSN in healthcare has alleviated the monitoring of patients and providing healthcare services. The WSN for heathcare is typically classified into three groups:

∙ In-body networks. This is used to communicate between the sensors that were implanted inside the patient’s body and a reader or receiver located outside of the patient’s body. For example, implantable pacemaker, implantable cardioverter defibrillator(ICD), or smart capsules that are used to transfer bio-information, which can measure in-body health attributes, and transmits to an external device or reader.

∙ On-body networks. This is used to communicate between the sensors that were attached on the patient’s body and the data gathering devices. For example, wrist watches to measure pulse rates, bio-shirts, and ring sensors that are attached to the patient’s body to transfer the sensed health attributes for processing and analysis.

∙ External networks. This is used to communicate between sensor nodes, actuators, local processing nodes, and remote servers. home and mobile healthcare that communicates between sensors, devices or a location processing unit that can communicate with a remote server. This involves both wireless and mobile networks.

Fig. 1 depicts bio-sensors gathering healthcare information from an elderly and forwards the collected healthcare information to a user device that acts as the WSN gateway. Nowadays, WSNs have been replaced by ubiquitous sensor networks(USNs) that is comprised of a convergence between application technologies for sensing, networking, and computing which are context-aware and knowledge-based driven.

Fig. 1.

WSN in healthcare

2.2 Ubiquitous Computing in Healthcare

Ubiquitous computing refers to a concept that has made computing becomes available at anytime and anywhere[9][10]. It is usually comprised of an interconnected things and inexpensive computers that supports everyday functions and chores in an automated fashion.

Ubiquitous computing has been integrated with healthcare and utilizes a large number of physiological and environmental sensors and actuators in monitoring and supporting the patient’s medical and physical conditions in real-time[7][8]. The u-Healthcare system employs the Wireless Sensor Network(WSN) technology in gathering and measuring information on bodily and environmental conditions such as temperature, blood pressure, heart beat rate, pulse rate, blood and urine chemical levels, breathing rate and volume, activity levels, and other physiological attributes that are significant in the diagnosis and treatment of the patient’s health conditions.

These ubiquitous and tiny sensors can be implanted in the patient’s body, worn on, or installed within the patient’s living or working environment. The USN actuators on the other hand trigger medical or healthcare events and actions such as releasing of a particular amount of medicine into the patient’s bloodstream, electrical stimulation of brain areas, triggering an alarm for a patient to take his/her medication, and alarming the healthcare staff or doctors with an emergency. The sensors and actuators provides the capability for the healthcare to be able to monitor in real-time the health conditions of patients and be able to design and implement intervention or diagnostic programs to improve the patients conditions and wellbeing. These devices were initially intended for remotely monitoring patients in order to provide healthcare advices and tips without physically visiting healthcare facilities which were particularly beneficial for mobility-impaired patients and many older people. Moreover, the technology becomes more of a self-monitoring and care system for most individuals and not only intended for the elderly and patients with chronic diseases.

The USN has evolved from WSN(i.e., the previous generation of sensor networks) that has converged with the ubiquitous computing technology. The USN technology has enabled the functionalities of WSN to become available at anytime and anywhere[11]. It does not focus only on sensor node networking and node system development but covers a wider spectrum in terms of user- and service perspective and deals with both lower-layer and higher-layer sensor network issues. Information in USN were acquired using context awareness through detecting, sensing, measuring, storing, processing, analyzing, and integrating physiological and environmental conditions that were collected from sensor nodes deployed geographically in the environment or embedded on anything. The USN technology deals with both the network installation view and the application services view.

For patients with cognitive impairments, more intensive support from their family members and healthcare personnel were required. Thus, u-Healthcare technologies can alleviate these services to provide real-time patient monitoring and deliver long-term healthcare programs such as taking proper diet and exercises.

In addition, u-Healthcare technologies and systems were also developed to support the activities of healthcare workers and doctors. Patient record systems were developed to support healthcare workers which are capable of automatically updating the information gathered from patients based on their current context, support for improved information flow between shift changes of healthcare workers, and robust transmission of real-time information from accident scenes to healthcare facilities. For doctors, surgery training systems were developed to support doctors’ training and simulation exercises.

2.3 Wireless Communication Technologies and Mobile Computing

The evolution of wireless communications has delivered an “anytime and anywhere connectivity”. The continuous enhancement of RFID and Wi-Fi standards and the emergence of 4G networks(now 5G networks) have facilitated the anytime and anywhere access to wireless and mobile communications[12]. The continuous developments in mobile and wireless computing will enable a seamless and robust real-time patient monitoring to continuously collect health-specific metrics from the deployed biomedical sensors and devices in the patients’ homes and other environmental settings outside the healthcare facility. Healthcare monitoring of patients typically measures and gathers patient’s physiological and environmental attributes(through context awareness) and forwards these healthcare information for storage, analysis, and processing by healthcare professionals which will be the basis for providing diagnosis or treatment programs.

Healthcare applications and services strictly imposes end-to-end system reliability and data delivery. Seamless and robust wireless network and mobility management are required to support the real-time and in-time transmission of healthcare information. Thus, optimizations on mobility management protocols are necessary and so is the emergence of 5G networks.

2.4 Augmented Reality/Virtual Reality (AR/VR)

Augmented reality(AR) refers to a technology that augments virtual elements into the real world. These virtual elements are overlaid into the real world to provide an interactive experience enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities such as visual, auditory, haptic, somatosensory, and olfactory. On the other hand, virtual reality(VR) refers to a technology where a virtually simulated experience can be similar to or completely different from the real world. A person interacts with the virtual environment through an artificial three-dimensional(3D) environment with the aid of special devices such as goggles, headsets, gloves, etc.

Both AR and VR technologies provide significant contributions to healthcare as both responds to real-time changes such as how a patient moves or perform his/her daily activities[13]-[15]. AR and VR in healthcare can assist surgeons with a safer and more efficient surgical planning. VR can also be used to educate patients before undergoing to surgeries.

In addition, AR technologies were integrated in medical training where an AR tool can display an internal view of the body to study its anatomy. Gesture-sensitive interfaces for these applications allow users to interact immersively with the representation.

2.5 Artificial Intelligence in Healthcare

The utilization of artificial intelligence(AI) technologies and machine-learning algorithms to mimic human cognition in the analysis, interpretation, and comprehension of complicated medical, mental, and healthcare data has been rapidly emerging. AI provides a healthcare system with the capability of making sound decisions and conclusions through computer algorithms based on the processed input data(i.e., collected healthcare data based on context). Through machine learning and deep learning algorithms, AI can also enhance the healthcare’s ability in gathering healthcare data, process it, and provide a well-defined conclusion or output to be presented to the healthcare professionals and the patients. AI algorithms must be trained using extensive amounts of healthcare input data to become more sophisticated and effective in giving predictions.

AI in healthcare provides a capability to analyze the relationships between prevention or treatment techniques and patient outcomes[16]. AI programs are applied to healthcare services such as diagnosis processes, development of proper treatment procedures, drug development, personalized medicine, patient monitoring and care, and wellbeing management.

AI in healthcare has revolutionized the healthcare systems and has provided significant improvements such as medical imaging, automated clinical decision-making, diagnosis, prognosis, and more[17].

2.6 Comparative Analysis among the Converging Technologies

The comparative analysis on the significant converging technologies highlighting their respective advantages and contributions to the future AR/VR-based u-Healthcare systems is depicted in Table 1.

Table 1.

Comparative analysis among the converging technologies

3.1 Medical Training Feature

The integration of AR and VR reality technologies in healthcare has provided a virtual experience that takes a person inside a human body in order to view and access body areas and parts that in real world would be impossible to reach. Traditional medical training were focused on the examination of cadavers which provide a lot of disadvantages as compared with the examination of a live patient body. With AR and VR, healthcare professionals can view and access every detail of any patient body part through 360 degree computer-generated imagery(CGI) reconstruction and can manipulate and create a variety of training scenarios that mimic common and actual surgical operations. The CGI models of the patient body parts were combined with VR environment to provide an immersive and interactive surgical training experience for surgeons and healthcare professionals.

Companies such as Medical Realities[21] and Osso VR[22] offers sophisticated surgical training and assessment platform that provides full simulations in delivering next-generation of immersive medical training and education. Fig. 2 depicts Osso VR’s technology being utilized in Newcastle Surgical Training Centre.

Fig. 2.

Osso VR's technology being used in newcastle surgical training center[23]

3.2 Treatment Administration Feature

The treatment administration feature for AR/VR-based u-Healthcare system can include various services that could guarantee an efficient and proper administration of diagnostic and treatment programs to patients and thus, warrant for faster recovery of patients and alleviate their wellbeing.

AR and VR technologies in healthcare can provide patient education through allowing them to view the inside of their human body. This enables them to properly understand their treatment and coordinate properly with healthcare professionals in planning in advance their surgical operations and increase their healthcare service satisfaction.

AR and VR technologies assists in robotic surgery that is performed by a robotic device being controlled by a human surgeon. This feature results in fewer risks of complications while performing the surgical operation and the procedure can be performed faster. The robotic device has greater accuracy, that means, it allows for smaller incisions, reduced blood loss, thus, results in faster patient recovery.

AR and VR technologies in healthcare also alleviates the capabilities of mental health and psychological therapies. Robust simulations can be created for patients to overcome their psychological difficulties through virtual experiences. For example, whenever patients with psychological difficulties such as going up on tall buildings or coming to crowded places, there will be no need for a therapist to handle the therapy physically. Mental health and psychological therapies will be administered through virtual experiences. This can also be administered to conjure the harrowing events behind post-traumatic stress disorder(PTSD). In addition, AR and VR technologies can be used to suppress memory loss and provide support for autistic children in performing their daily activities.

AR and VR technologies in healthcare has also been proven to be effective for pain management and physical therapies[24]. Physical therapists may administer virtual rehabilitations with exercise routines which can be accessed and performed by patients while they are confined in their homes. AR/VR-based tasks and scenarios within a controlled virtual environment can be very effective in making the recovery time more faster. Moreover, an interactive and immersive virtual experience can be a distraction and may reduce pain levels for patients that have undergone surgical operations. The virtual experience can provide more fun and keeps the patient immersed to allow them to focus on recovery even if it can take a longer period.

3.3 Disease Awareness Feature

AR and VR technologies can provide the healthcare system with a feature that presents diseases and wellbeing awareness to the society. For example, the daily struggles that patients suffering from Alzheimers disease or Parkinsons disease can be simulated in a virtual experience to raise awareness among healthcare professionals and other users. The persons that uses the virtual simulation may experience awkward and embarrassing moments when then come in contact with other people or in performing the given tasks.

Thus, with proper disease awareness, healthcare professionals and doctors can carefully plan a long-term healthcare program and treatment as well as cater the daily care needs of patients for a particular disease. For other people, experiencing the virtual simulation of diseases can make them decide in advance their long-term healthcare options, healthy lifestyle, and wellbeing[25].