Introduction
Augmented Reality (AR) is reality as perceived by men, enhanced by additional information that overlaps it but does not replace it. The non-replacement of truth is the concept behind the substantial difference with virtual reality (VR), in which a PC entirely generates the experience, and the individual is wholly embedded in it.
The additional information is generated by a computer and conveyed to the user through different means. The most widespread means to experience AR are external displays. A general example might be the offside line added to the broadcasted images of a football match or the augmented navigation through our cities’ streets with a smartphone in our hands.
In recent years we have seen an impulse in developing new technologies that would allow a more ergonomic conveyance of this information
and its integration with an egocentric point of view of reality.
Head-Mounted Displays (HMD) were developed in an attempt to satisfy this need. HMDs are displays that can be mounted on the operator’s head and convey the data required to create the augmented experience. The subcategory of HMDs that has undergone the most remarkable development is that of HMD Optical See-Through (OST), HMDs that use semi-transparent screens, allowing the operator to detect reality through the transparent glass. At the same time, the additional information generated by the computer is projected on it. In this field, the market leader in Microsoft, with its HoloLens, now at its second generation and boasting the largest number of applications and software in development. An obvious advantage of OST-HMD is that they allow us to experience AR Abstract
Introduction: Many technologies have been proposed in these years to improve the quality and results of orthopedic surgery. The most common are robot-assisted surgery, navigation, patient-specific instrumentation, and customized implants. In the last years, augmented reality has been used in many settings, and some companies now propose its improvement in surgery. This narrative review aims to describe the state of the art and prospects of augmented reality in orthopedic surgery.
Methods: A narrative review of the current literature was performed in September 2020 to find articles focused on augmented reality in orthopedic surgery in Scopus and PubMed/Medline with the following keywords: (augmented reality) AND (orthopedic). Inclusion criteria were articles published in the last 20 years, reviews, clinical and educational articles. The authors collected the data described in these articles and analyzed it to find the most relevant information on this topic.
Results: One hundred forty-five articles on augmented reality in surgery were found. The majority of these studies were focused on the potential use of this technology for young surgeons’ training and instruments’ accuracy.
Conclusions: Augmented reality is potentially a significant help for orthopedic surgeon’s practice and training. It is realistic to expect a healthy development of this technology in the coming years and the future use in operating fields to improve the precision and reproducibility of surgery.
Level of evidence: V
Keywords: Augmented Reality; Orthopedic surgery
Augmented reality in orthopedic surgery:
state of the art and future prospectives
Riccardo Compagnoni 1,2, Gianluca Coccioli 2,
Filippo Calanna 2, Paolo Ferrua 2, Pietro Randelli 1,2.
1) Laboratory of Applied Biomechanics, Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Mangiagalli 31, 20133 Milan, Italy; 2) 1° Clinica Ortopedica, ASST Centro Specialistico Ortopedico Traumatologico Gaetano Pini-CTO, Piazza Cardinal Ferrari 1, 20122 Milan, Italy.
Corresponding author: Gianluca Coccioli - Via Pini,3 20122 Milano - coccioli.gianluca@gmail.com
(2020) AMSUM 1(2):10. doi:10.13130/AMSUM/14955
A M S U M
Archivesof MedicineAnd surgery ofthe universityof Mil An
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International (CC BY-NC-ND 4.0)
Compagnoni et al., Augmented Reality in orthopedics. (2020) AMSUM 1(2):35-37.
36
while being hands-free: the AR can be navigated through the use of gestures or voice commands. OST-HMDs make the use of AR in surgery not only possible but also ideal.
Many technologies have been proposed in these years to improve the quality and results of orthopedic surgery. The most common are robot-assisted surgery, navigation, patient-specific instrumentation, and customized implants. In the last years, augmented reality has been used in many settings, and some companies now propose its improvement in surgery. This narrative review aims to describe the state of the art and prospects of augmented reality in orthopedic surgery. Materials and method
A narrative review of current literature has been performed to find articles focused on augmented reality in orthopedic surgery in Scopus and PubMed/Medline with the following keywords: (augmented reality) AND (orthopedic). Inclusion criteria were articles published in the last 20 years, reviews, clinical and educational articles. The authors collected the data described in these articles and analyzed it to find the most relevant information on this topic.
There are many possible applications.
Results
One hundred forty-five articles on augmented reality in surgery were found. The majority of these studies were focused on the potential use of this technology for young surgeons’ training and instruments’ accuracy.
Discussion
AR can be beneficial in surgical training. It can enhance the experience when the practice is performed on models and phantoms, making it more realistic or applied directly in the operating room. Ponce et al. [1] 2014 publishes a paper on a particular teaching experience: telementoring through the use of VIP (virtual interactive presence) in shoulder arthroscopy: the resident or attending surgeon
received instructions and clarifications by visualizing the teacher’s hands directly in the operating field. The experience was positively evaluated by all the staff involved.
The publication of Fotouhi et al. [2] describes an excellent example of AR used in the operating room. Although they did not use HMDs (the information was displayed on a screen away from the operative field), the experiment represents an excellent example of how AR can streamline the activity of the surgeon in the OR, providing a solution to the difficult task of envisioning in a three-dimensional reality a 2D XR image or CT scan. Using an RGBD cam (i.e., a camera that can detect the depth of field (D=depth), other than RGB colors) combined with intraoperative x-rays allowed the surgeon to have real-time feedback the three-dimensional positioning of the acetabular component of a total hip prosthesis.
Ortega et al. [3] in 2008 applied AR to trauma surgery through a (now obsolete) OST-HMD, projecting the patient’s pre and intraoperative X-rays into the operator’s field of view and thus demonstrating how an AR viewer can reduce the number of times the surgeon is forced to look at a distant screen. This allowed for a drastic reduction in the need for intraoperative X-rays for the benefit of the patient and all OR staff.
There are several problems to be solved before this technology can be seamlessly integrated into an operating theatre.
Condino et al. [4] describe the physical limits of human beings that clash with the use of the current generation’s HMD during surgeries. The focal length of modern HMDs is 2.0 meters.
Enhancing reality with 3D images superimposed on the patient’s anatomy is therefore impossible since the operator would not focus on both. Instead, it would be possible to display additional information positioned in a different place in the operator’s field of view so that the surgeon could visualize it by briefly
Compagnoni et al., Augmented Reality in orthopedics. (2020) AMSUM 1(2):35-37.
37
moving its focus from the operating area. The cost of implementing the technology (hardware, software, training) is another essential obstacle.
The lack of a finished and functioning product is the factor that more severely limits the expansion of the technology.
Although manufacturers announced their interest and commitment to the R&D of AR applied to surgery, there are currently no commercial products available that health care companies can purchase.
There are some software houses such as Medivis (medivis.com) and Novarad (novarad.net/opensight) that are working on software applicable to surgery through Microsoft’s HoloLens 2, but they currently are in a developmental phase of their products (although already FDA approved).
Conclusion
Nowadays, applying AR to the operating room would only be possible through experimentation conducted by surgeons and AR experts. A way to do this would be to get the engineering departments of universities involved so that surgeons and IT experts could devise new trials and new advances in research that could push forward the future expansion of this technology. Augmented reality is potentially a significant help for orthopedic surgeon’s practice and training. It is realistic to expect a healthy development of this technology in the coming years and the future use in operating fields to improve the precision and reproducibility of surgery.
Declaration of interests: None declared. Funding: No funding to declare.
References
1. Ponce BA, Jennings JK, Clay TB, May MB, Huisingh C, Sheppard ED. (2014) Telementoring: use
of augmented reality in orthopaedic education: AAOS exhibit selection. J Bone Joint Surg Am. 96(10):e84. 2. Fotouhi J, Alexander CP, Unberath M, Taylor G, Lee SC, Fuerst B, (2018) Plan in 2-D, execute in 3-D: an augmented reality solution for cup placement in total hip arthroplasty. J Med Imaging (Bellingham) 5(2):021205.
3. Ortega G, Wolff A, Baumgaertner M, Kendoff D. (2008) Usefulness of a head mounted monitor device for viewing intraoperative fluoroscopy during orthopaedic procedures. Arch Orthop Trauma Surg.128(10):1123-6. 4. Condino S, Carbone M, Piazza R, Ferrari M, Ferrari V. (2020) Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks. IEEE Trans Biomed Eng. 67(2):411-419.
5. Laverdière C, Corban J, Khoury J, Ge SM, Schupbach J, Harvey EJ. (2019) Augmented reality in orthopaedics: a systematic review and a window on future possibilities. Bone Joint J. 101-B(12):1479-1488
