Clinically Driven Collaborative Temporal Bone Surgery Training System
This project concentrated on the surgical problem of drilling of the temporal
bone in the skull for ear surgery. We worked with otologists from the University
of Melbourne to develop a collaborative virtual environment for teaching this
complex task. We concentrated specifically on the issues of collaboration, that
is, how an instructor could share the environment with a student and offer
useful training. We developed interactive models of the temporal bone, derived
from CT scans of actual bone samples, that enabled the instructor to point out
the vital relationships between the various anatomical features, and the
sequence of steps that must be undertaken to complete the procedure safely. The
system used haptic workbench
technology, and included graphic and haptic rendering of the interactive
drilling of the volumetric bone models. The system was assessed in a clinical
trial in November 2004, where we found significant evidence of positive training
transfer (preliminary results have been published, a more detailed publication
will follow). The intellectual property developed for the research system has
now been licensed to an Australian medical technology company,
Medic Vision, to develop into a
training product.
Collaborative Gallbladder Training
We developed a concept demonstrator that showed how collaborative haptic virtual
environments could be used to teach the anatomy and surgical approach for
laparoscopic cholecystectomy (removal of the gallbladder through a keyhole
incision). The system demonstrated how an instructor could link to a trainee,
who could be in another room or even another country, and guide them through the
steps necessary to conduct the procedure. As well as including interactive
models of representative patient anatomy, the system also featured new
approaches to remote teaching, including annotation of the model in 3D, joint
haptic manipulation of deformable objects, haptic hand-on-hand guiding, surgical
instrument vapour trails and shared access to pre-recorded video segments and
other media such as x-rays. The system was showcased in a live demonstration at
the SimTecT conference in 2004, linking conference attendees in Canberra,
Australia, with surgical trainers at Stanford University School of Medicine,
USA.
Schematic of haptic interaction with anatomical
model
Gall-bladder surgery trainer showing both student's and instructor's graspers
manipulating the position of the bladder and bile duct
Gall-bladder surgery trainer showing diathermy to cut the retaining membrane
3D surgical video across the Pacific
In March 2006, Sydney surgeons enjoyed a two hour window into the operating room
at Stanford University Hospital from the comfort of the Director's Meeting Room
at our Marsfield site. They discussed the operation beforehand with the surgeon,
went with him into the Operating Room to be briefed on the placement of keyhole
surgical instruments then followed him in glorious 3D into the patient as he
gave running commentary on the steps of the surgery. We are one of few research
groups to have achieved high-quality real-time 3D video of surgery over the
internet in a practical teaching scenario, and we showed that this type of
presentation is valuable right across the range of surgical experience.
The 3D display was the centrepiece of the event. The larger-than-life display
showed the abdominal organs in stunning clarity as the surgeon inspected them
for disease. At one point he needed to remove the appendix, and after explaining
that there were several different techniques for doing this he asked the
audience which technique they would like to see applied! The display technology
used two powerful data projectors projecting through polarised glass filters
onto a specially surfaced screen (to preserve polarisation of the reflected
light). The audience wore simple plastic polarised glasses.
This was a team effort. Extensive background work between the Canberra team and
our colleagues at Stanford got the basic software and logistics in place. The
surgeons (Pat Cregan in Sydney, LeRoy Heinrichs and Camran Nezhat in Stanford)
developed the teaching material and organised hospital, patient and audience.
Our Sydney colleagues played a major role in setting up and fine-tuning the
system at Marsfield, and in liaising with AARNet to switch to serious bandwidth
across the Pacific. Dr Nezhat and his surgical team performed the operation.
Augmented Reality
The team has combined augmented reality technology (using the ARToolKit
from the Human Interfaces Technology Laboratory) with its own haptic and
collaborative technologies to develop concept demonstrators. These combine the
features of the collaborative haptic workbench with the concept of face-to-face
discussions about a virtual object placed on a tabletop between two people.
Ongoing research is looking at ways of using this capability to enhance
long-distance interaction between people who are jointly solving a problem or
performing a task.
Augmented Reality brain anatomy tutor showing horizontal slice from the
Virtual Human data set.
|
Augmented Reality display of silver museum artefact in a story-telling
context
|
| |
|
For more information on AR Toolkit, visit
http://www.hitl.washington.edu/artoolkit/.