Intelligent Virtual Environments for Surgical Training

Coordinator: Dr. Dorothe Poggel (HWK)

Speaker:

Prof. Dr. Peter Haddawy, Faculty of ICT, Mahidol University, former HWK Fellow

Members:

  • Prof. Dr. Christian Freksa, Bremen Spatial Cognition Center, University of Bremen
  • Prof. Dr. Johannes Schöning, Bremen Spatial Cognition Center, University of Bremen
  • PD Dr. Holger Schultheis, Bremen Spatial Cognition Center, University of Bremen
  • Prof. Dr. Ron Kikinis, AG Medical Image Computing, University of Bremen, and Fraunhofer MeVIS
  • Prof. Dr. Rainer Malaka, Digital Medial Lab, University of Bremen
  • Prof. Dr. Gabriel Zachmann, AG Computer Graphics and Virtual Reality, University of Bremen
  • Prof. Dr. Michael Beetz, Institute for Artificial Intelligence, University of Bremen
  • Prof. Dr. Enkelejda Kasneci, AG Perception Engineering, Dept of Computer Sci, Univ of Tübingen
  • Prof. Dr. Siriwan Suebnukarn, Faculty of Dentistry, Thammasat University (former HWK Twin fellow)
  • Dr. Mores Prachyabrued, Faculty of ICT, Mahidol University
  • Prof. Dr. Paphon Sa-ngasoongsong, Faculty of Medicine, Ramathibodi Hospital, Mahidol University
  • Dr. Myat Su Yin, Faculty of Information and Communication Technology, Mahidol University

Associated PhD-Students

  • Tina Vajsbaher, Bremen Spatial Cognition center, University of Bremen
  • Daniel Diethei, Human Computer Interaction Lab, University of Bremen
  • Narumol Vannaprathip, Mahidol University

Duration:

May 1, 2018 – April 30, 2021

Background

The teaching of surgical skills, which has traditionally followed an apprentice style approach, is now facing a number of challenges that are driving medical schools to seek alternatives. Increasing enrollments combined with constraints on the time of expert surgeons means that students often do not receive as much supervised training as would be desirable.  Students end up spending a significant portion of their training time carrying out unsupervised practice. Even when assessment is carried out, it is by nature subjective, lacking sufficient standardization. 

Surgical competence requires psychomotor skill related to a surgeon’s clinical knowledge, dexterity, and the use of surgical equipment, and it requires cognitive skills such as decision making.  While it is convenient for purposes of analysis and even teaching to separate the two categories of skills, there is, in fact, a complex interplay between them in terms of a dynamic link between perception, comprehension, and action.  The high level of the physical precision required and the dynamic nature of the decision making environment place high and at times conflicting demands upon the attention of the surgeon.  In building intelligent systems for teaching of surgery it is therefore important to address psychomotor skill and cognitive skill in concert.  

Goals

Our objective is to explore the potential of the integration of techniques from Virtual Reality and Intelligent Tutoring Systems to increase the effectiveness of surgical training by providing personalized instruction, assessment, and formative feedback in a form and on a scale not possible in the physical world.  While most work on virtual training systems to date has focused either on psychomotor or decision making skills, our approach will be to study and build prototype intelligent systems for jointly teaching both categories of surgical skill.  Specifically, we will address the following questions:

  • How to represent surgical procedures in such a way that we capture the symbolic level description of the procedure as well as the kinematics and the relation between them.  This is necessary in order to be able to effectively analyze procedures and to communicate about them.
  • Understand teaching strategies used by expert surgeons at a level of detail that they can be automated.
  • How to diagnose the causes of student errors, whether cognitive, psychomotor, or both so as to be able to provide appropriate feedback.
  • How to automate the teaching process, with particular emphasis on creating rich dialog with the student about causal mechanisms and on coaching and guiding rather than telling.
  • How to build effective virtual reality environments and understand the impact of different dimensions of realism on teaching effectiveness.
  • Investigate which formative feedback mechanisms are most effective in terms of immediate and long-term learning gains.