Bitte aktualisieren Sie Ihren Browser zur korrekten Anzeige dieser Webseite.

Knowledge Visualization

Journey to the Inner Ear

Visual Design of Robot-Assisted Surgery for the Hearing Impaired

How can designing audience-specific visualizations improve the user experience of a novel robot-assisted surgical platform? What visual aspects are important to improve accuracy, orientation, and safety for the surgeon user? What visual aspects are important to reduce fear and improve trust for patients about to undergo the surgery?

Background

Five percent of the world is deaf or hard of hearing. The cochlea is the organ that allows us to hear, and thousands of cochlear implantati- ons are performed each year to restore the sense of hearing. Because ear anatomy is difficult and the surgery is technically challenging, 30–50% of cases are not successful. With modern technology, surgeons like myself now have a solution – surgical robots. Robots can overcome human limitation, making the surgery more effective, less invasive, and safer for the patient.

Aims

Programming the surgical robot requires creating 3D models from patient-specific CT scans. The aim of this project was to explore the visual needs of the various audiences of this novel surgical system.

Methods

After using Amira to generate 3D models from CT scans, I then used Cinema 4D to experiment with various visual styles. In collaboration with engineers and surgeons, I conducted focus groups, in-person interviews, and surveys to test the surgeon user experience of lifelike compared to the baseline unnaturally-colored visualizations. Quantitative studies were performed test the effect of lifelike visualizations on ability to orient and identify anatomical structures. Interviews and surveys were used to explore how best to use visualizations to guide the patient experience of undergoing robot-assisted surgery.

Results and Conclusions

Lifelike visualizations improve the user experience for surgeons regarding accuracy, orientation, and satisfaction, while graphical visualizations can enhance patient understanding and trust in a robot-assisted surgical system.

Catherine Tsai
MA-Diplome 2019


Mentoring
Fabienne Boldt, Simon Tschachtli

Cooperation partner
ARTORG Center for Biomedical Engineering, University of Bern

Inner ear structures: Life-like 3D reconstruction of the cochlea, malleus, incus, stapes, and facial nerves from a patient‘s CT scan. Transparency allows us to see the cochlear implant electrode threading into and coiling within the cochlea.
Inner ear structures: Life-like 3D reconstruction of the cochlea, malleus, incus, stapes, and facial nerves from a patient‘s CT scan. Transparency allows us to see the cochlear implant electrode threading into and coiling within the cochlea.
Skull and cochlea transplant: Life-like 3D reconstruction of a patient‘s skull with cochlear implant and blue fiducials (screws). Fiducials allow the robot to capture the movement of the patient‘s head through optical tracking.
Skull and cochlea transplant: Life-like 3D reconstruction of a patient‘s skull with cochlear implant and blue fiducials (screws). Fiducials allow the robot to capture the movement of the patient‘s head through optical tracking.
Patient-specific 3D ear and skull models are generated from the pixels of a computed tomography (CT) scan using Amira software.
Patient-specific 3D ear and skull models are generated from the pixels of a computed tomography (CT) scan using Amira software.
Using Cinema 4D, various visual themes are created. Models are shared using Sketch- fab to test the ability of the visualizations to improve accuracy of identifying structures and ability to orient structures.
Using Cinema 4D, various visual themes are created. Models are shared using Sketch- fab to test the ability of the visualizations to improve accuracy of identifying structures and ability to orient structures.
After the final designs are created, engineers at ARTORG Center program them into Otoplan (see below).
After the final designs are created, engineers at ARTORG Center program them into Otoplan (see below).
A tablet for surgical planning: Otoplan is a software on a tablet device that allows surgeons to generate 3D ear models from patient CT scans and plan the trajectory of the robot drill. This master‘s project optimizes the visual interface for the surgeon user.
A tablet for surgical planning: Otoplan is a software on a tablet device that allows surgeons to generate 3D ear models from patient CT scans and plan the trajectory of the robot drill. This master‘s project optimizes the visual interface for the surgeon user.