VR Experience  •  2017 Aesculap OrthoPilot®
Virtual Surgery Training

The OrthoPilot® surgery navigation system by Aesculap B. Braun supports surgeons during the implantation process of knee and hip prosthetics.

At NMY I was the lead designer on a project team that developed a gamified VR simulation of a knee surgery guided with the OrthoPilot®, that was used for marketing and training purposes.

Client: Aesculap / B. Braun
With: NMY Mixed-Reality Communication GmbH
My role: Concept, Art Direction, UX/UI Design, UI & Animations in Unity

About the project

To better communicate the novel and user friendly surgical procedures introduced with the OrthoPilot® system in in-house demos and on trade shows, B. Braun Aesculap was searching for a product presentation that went beyond mere PowerPoint slides or videos. The solution was a VR simulation that had to both accurately represent the various steps of the surgery as well as enable users, that had never used a VR headset before, to complete the whole procedure during a limited time frame. The audience should also learn about the features and benefits of the system, while still having some fun with the experience. To achieve this goal, the surgery process was drastically simplified for the VR experience and game mechanics were used to guide the users and encourage them to complete the simulated tasks with as much precision as possible.

Insight  •  Design Process Translating a complex surgery workflow into a gamified and intuitive simulation.

After an initial workshop with the OrthoPilot® product team to understand the system as much as possible, I set out to create first sequence diagrams and scribbled a storyboard to define the rough scope and segments of the experience.

In close collaboration with the development team, ideas for interactions and game mechanics were tested with interactive prototypes in VR as quickly as possible, which helped to gather client feedback and get a better understanding of what could be done in VR early on.

Based upon a more detailed storyboard and wireframes, I designed the UI and implemented it in Unity where I also worked on a lot of the animations used in the experience.

Insight  •  Challenges & Learnings Finding the right balance between simplified gamification and medical accuracy.

The manipulation of the surgery tools and patient leg, necessary to represent the procedure accurately, was quite complex compared to usual interaction with objects or UI in VR. That made it quite a challenge to translate the process into easy to understand, gamified VR interactions.

The trick was to provide a lot of user guidance, leveraging audiovisual and haptic feedback and the possibility to display information exactly at the point in space, where it is needed. In some cases it also helped to limit the degrees of freedom with which an object can be manipulated.

In addition to body storming and scribbles, early interactive prototypes were essential in finding solutions that worked well and conveyed a feeling of guided control, representing the value of support the OrthoPilot® aims to give surgeons in reality.

Information that would normally be communicated via the OrthoPilot®'s screen is displayed as a spatial UI around the knee in VR.

Audiovisual and haptic feedback served as a spatial guidance system for custom interactions and helped users to work more precisely, even on the first try.

A compass-like pointer on the controller was used to show users where to look for the currently needed tool or object.

Additional guidance and product information was provided via hints on labels or tool-tips.

New interactions were first explained via ghost animations and could be tested out before they were required to be used in a task.

A precision score and leader board encouraged users to challenge other players and improve their own performance.

You can download a reduced version of the experience including the first task of the surgery, the palpation, as a VR mini-game for Vive headsets on Steam.

Download on Steam

More projects

iPad AR App  •  2018

Aesculap Aeos®

AR Product Guide

VR Experience  •  2016

B. Braun

Future Operating Room

© Joris Klause