Injury, Int. J. Care Injured 46 (2015) 2212–2216
Contents lists available at ScienceDirect
Injury
journal homepage: www.elsevier.com/locate/injury
Validating Touch SurgeryTM: A cognitive task simulation and rehearsal
app for intramedullary femoral nailing
Kapil Sugand *, Mala Mawkin, Chinmay Gupte
MSk Lab, Imperial College London, Level 7 East, Charing Cross Hospital, Fulham, London, W6 8RF, UK
A R T I C L E I N F O
A B S T R A C T
Article history:
Accepted 2 May 2015
Background: The role of simulation in orthopaedic surgical training is becoming increasingly evident, as
simulation allows repeated sustained practice in an environment that does not harm the patient.
Previous studies have shown that the cognitive aspects of surgery are of equal if not greater importance
in developing decision making than the practical aspects.
Aim: To observe construct, content and face validity of four IFN modules on a cognitive simulator, Touch
SurgeryTM.
Methods: 39 novices and 10 experts were recruited to complete four simulation modules on surgical
decision-making that represented the procedural steps of preparing the patient and equipment,
inserting and locking an intramedullary femoral nail. Real-time objective performance metrics were
obtained, stored electronically and analysed using median and Bonett-Price 95% confidence intervals
from the participant’s primary attempt to assess for construct validity. The median score of a post-study
questionnaire using 5-point Likert scales assessed face and content validity. Data was confirmed as nonparametric by the Kolmogorov–Smirnov test. Significance was calculated using the Mann–Whitney U
test for independent data whilst the Wilcoxon signed ranked test was used for paired data. Significance
was set as 2-tailed p-value < 0.05.
Results: Experts significantly outperformed novices in all four modules to demonstrate construct
validity (p < 0.001). Specifically, experts scored 32.5% higher for patient positioning and preparation
(p < 0.0001), 31.5% higher for femoral canal preparation (p < 0.0001), 22.5% higher for proximal locking
(p < 0.0001) and 17% higher scores for distal locking and closure (p < 0.001). Both cohorts either agreed
or strongly agreed that the graphics, simulated environment and procedural steps were realistic. Also,
both cohorts agreed that the app was useful for surgical training and rehearsal, should be implemented
within the curriculum and would want to use it to learn other surgical procedures.
Conclusion: IFN on the Touch Surgery app demonstrated construct, face and content validity. Users can
demonstrate cognitive competencies prior to performing surgical procedures in the operating room. The
application is an effective adjunct to traditional learning methods and has potential for curricular
implementation.
ß 2015 Elsevier Ltd. All rights reserved.
Keywords:
Intramedullary femoral nailing
Simulation
Touch surgery
Training
Orthopaedic
Trauma
Education
Introduction
Orthopaedic trauma is a leading contributor to the global
burden of non-communicable disease. One of the commonest
trauma scenarios faced by orthopaedic surgeons is femoral
fractures including sub-trochanteric and diaphyseal configurations. Both can be definitively treated with closed reduction on the
traction table and internal fixation using an intramedullary nail.
* Corresponding author. Tel.: +44 7773642813.
E-mail addresses: [email protected], [email protected] (K. Sugand),
[email protected] (M. Mawkin), [email protected] (C. Gupte).
http://dx.doi.org/10.1016/j.injury.2015.05.013
0020–1383/ß 2015 Elsevier Ltd. All rights reserved.
Total incident rate for femoral diaphyseal fractures is 10 per
100,000 person-years and the incident rate ratio between men to
women is 0.9 with an increasing prevalence [1,2].
Orthopaedic surgery is traditionally learnt through repeated
practice on patients and proctorship [3]. In recent years this
process is changing, as a result of multiple factors including
increasing subspecialisation and complexity of procedures and
devices, a reduction in training hours in North America and Europe
[4,5], and concerns regarding patient safety [6].
Simulation in surgery has become an important component of
the training process, providing a way for trainees to practise
operative tasks in a protected environment without putting
patients at risk. When this is extended to just before an operation
K. Sugand et al. / Injury, Int. J. Care Injured 46 (2015) 2212–2216
or combined with patient specific cases it can be referred to as
‘surgical rehearsal’. The concept of rehearsal is relatively foreign in
surgery compared to other professions. Yet, studies have shown
that rehearsing for a procedure before performing it can decrease
both the operating time and errors [7].
Orthopaedic simulation consists drybones or cadaveric specimen. Limitations include access to dry and wetlabs, shortage of
cadavers and the high expense of teaching facilities and equipment.
Multimedia software and virtual reality (VR) simulators may
overcome numerous limitations but expenses remain a significant
barrier to widespread adoption. Although mechanical and VR
constructs allow surgeons to acquire psychomotor and technical
skills, they neglect an emphasis on important non-technical skills
like cognitive decision-making. Decision-making contributes to
demonstration of clinical and non-technical competency [8,9].
The Touch Surgery application (Kinosis Limited, London, UK) is
a unique cognitive task simulation and rehearsal (CTSR) platform.
It focuses exclusively on the cognitive decision-making proponents
of surgical procedures. Touch Surgery is a simulation app that uses
cognitive task analysis (CTA) as a fundamental framework, and
combines this framework with VR to simulate procedures on their
mobile smart devices [10].
The primary objective of this study was to assess for construct
validity of the intramedullary femoral nailing (IFN) modules on
Touch Surgery using objective metrics. Secondary objectives
observed for face and content validity using a subjective
questionnaire. Research Ethics Committee reviewed and granted
ethical approval for this study (MEEC1213-17).
Methods
Touch surgery application
The Touch Surgery application is an interactive and immersive
VR multimedia software available on mobile and smart devices. It
offers a step-by-step manual to complete orthopaedic operations.
Test modules depend on completing manual steps (e.g. drilling,
reaming, incising and screwdriving etc). The other component is
assessing decision-making with single best answer questions (out
of four options) in each step of the operation. The IFN procedure
was authored by the consultant surgeons at the Musculoskeletal
Lab (MSk) at Imperial College London, UK, and developed at Touch
Surgery Labs. The procedure is divided into four separate modules
consisting of (i) patient positioning and preparation, (ii) femoral
canal preparation, (iii) proximal locking, and (iv) distal locking and
closure. Participants used the app pre-loaded on a Samsung tablet
(Samsung, Seoul, South Korea).
1
2
3
4
5
2213
Very poor/very useless.
Poor/useless.
Acceptable.
Good/useful.
Very good/very useful.
Face validation was assessed by both cohorts to see if the app
represented a useful training and assessment tool [11]. Content
validation was assessed by the expert group only by asking if the
accuracy and flow of steps represented the IFN procedure, [11]. The
acceptability was assessed by asking if participants would use the
app as a training tool to learn new operations and to rehearse an
operation before a procedure.
Data collection
The implementation of the study followed a rigid step-by-step
program to ensure a standardised experience for participants as
shown in Figs. 1 and 2. All data was collected online in real-time and
retrieved on the lab server. Points were scored from making correct
decisions (Fig. 2A), appropriate swipe interactions (Fig. 2B) and from
the time taken to complete each step. The app calculated total scores
as a percentage and there was no negative marking (Fig. 3).
Data analysis
All objective data were recorded as median with Bonett-Price
95% confidence intervals. Data was confirmed as non-parametric by
the Kolmogorov–Smirnov test. Analysis was performed using the
Mann–Whitney U test for independent data whilst the Wilcoxon
signed ranked test was used for paired data. A result was deemed
significant when a two-tailed p-value was less than 0.05.
Results
Study participants
48 medical undergraduates were initially recruited to the study.
2 of these were excluded from the study as they had used Touch
Surgery before. 7 did not attend on the day of the study leaving 39
(81%) novice participants in total. The median age of the novice
group was 19 years of age, with 46% males and 54% females. The
median completed years of study was 1 year. 10 experts consisting
of Orthopaedic Fellows and Attendings were recruited from the
surgical training circuit. The median age of the expert group was
33 years of age and 100% males.
Objective performance metrics
Participants
A total 58 participants from Imperial College London volunteered
to participate in the validation study. The novice group (n = 48)
consisted of medical undergraduates. Inclusion criteria for the
novice group was to have never observed or performed an IFN
procedure. The expert group consisted of senior Orthopaedic
surgeons (senior trainees and consultants). Inclusion criterion for
this group was the ability to perform the IFN procedure independently. Exclusion criteria for both cohorts consisted of dropping out
of the study and having previous exposure to the Touch Surgery app.
Study questionnaire
Both cohorts completed a pre-study questionnaire to ensure the
inclusion criteria were met. A post-study questionnaire asked the
participants to rate the app using a five-point Likert rating scale
where each score was defined as follows:
The experts significantly outperformed the novices in every
module; by 32.5% for the patient preparation module, 31.5% for the
femoral canal preparation module, 22.5% for the proximal locking
and 17% for the distal locking module (Table 1). There was also a
significant difference between males in both cohorts and gender
played no major part in the results.
Data from the feedback questionnaire were also collected
online. Mean values were calculated from the 5-point Likert scale
results (Table 2). Both cohorts rated the face validity, quality of
graphics, willingness to use the app, usefulness for preoperative
rehearsal as good or very good. Experts rated the content validity
as good too.
Discussion
Our study demonstrated construct, face and content validity of
the Touch Surgery simulation when applied to an IFN procedure.
K. Sugand et al. / Injury, Int. J. Care Injured 46 (2015) 2212–2216
2214
A: Novice cohort
Medical students recruited
n=48
Pre-study questionnaires
B: Expert cohort
Participant which had used
Touch Surgery excluded n=2
Participants lost to non-attendance
n=7
Pre-study questionnaires
Introduction and instruction
to participants
Introduction and instruction
to participants n=39
pre-module questionnaire
pre-module questionnaire
Completetion of module
Experts recruited (n=10)
Repeated for all 4 modules
Completetion of module
Post-module questionnaire
Post-module questionnaire
Post study questionnaire
Post study questionnaire
Repeated for all 4 modules
Fig. 1. CONSORT diagram on participants.
With regards to construct validity, novices achieved a median
scoring range of 41–60% whereas experts achieved a median
scoring range of 72–77.5% and this difference was significant. We
originally expected the difference to be even greater (i.e. experts
scoring even higher and novices lower). One explanation for this is
that experts do not necessarily follow the operation according to
steps outlined in the manual or textbook and have developed
habits to lead to an individualised routine but with the same
operative outcome.
The intramedullary femoral nail procedure was developed on
Touch Surgery using a framework of cognitive task analysis (CTA).
CTA was described in the 1980s as a revolutionary training method
Fig. 2. Snapshots of modules showing (A) multiple choice questions for decision-making and (B) digital swipe interaction to complete the procedure.
K. Sugand et al. / Injury, Int. J. Care Injured 46 (2015) 2212–2216
2215
Table 1
Comparison of performance metrics between Novice and Expert cohorts.
Metric
Novice median (95% CI)
Expert median (95% CI)
Percentage difference (%)
p-value
(i) Patient positioning and preparation
(ii) Femoral canal preparation
(iii) Proximal locking
(iv) Distal locking and closure
45.0
41.0
52.0
60.0
77.5
72.5
77.5
77.0
32.5
31.5
22.5
17.0
<0.0001
<0.0001
<0.0001
0.001
(42.7–47.3)
(37.7–44.3)
(48.9–55.1)
(55.5–64.5)
(73.2–81.8)
(66.5–78.5)
(68.3–86.7)
(67.5–86.5)
Fig. 3. Box plots showing objective performance scoring for all four modules.
which changed the way industry trained their professionals.
Companies found themselves shifting from using observational
learning and apprenticeships to CTA. Research on CTA demonstrated
it was a cost effective training method that could also significantly
reduce training time with some studies claiming that 50 h of training
correlated with 5 years of advanced job knowledge [10].
CTA has been used in highly technical fields to hasten the
acquisition of expert skill, and can improve learning and retention of
technical skills as well as decision-making in surgical procedures.
Therefore, it enables experts to articulate operative steps and
cognitive decisions in complex procedures as demonstrated by
Smink et al. using CTA for laparoscopic appendectomy [12], central
venous catheter placement and cricothyrotomy [13,14].
CTA has also been applied to rehearsal training in the sports and
music industries where evidence has shown that symbolic or even
mental rehearsal can improve performance and even, in some
Table 2
Comparison of median scores for content and face validity questionnaire between
Novice and Expert cohorts (max score = 5).
Questions
Novices
Experts
Face validity
Content validity
Would use the app
Useful for surgical training
Usefulness for preoperative rehearsal
Would use this app to learn more procedures
Want this as part of training curriculum
4
N/A
5
4
4
5
4
5
4
4
4
4
4
4
cases, to a degree better than physical rehearsal [15] [16]. Cognitive
task rehearsal has also recently demonstrated improve outcomes
on other virtual reality surgical simulator for laparoscopic
cholecystectomy [17] as well as technical skill performance of
conducting central venous catheterisation on patients [18].
The dissemination of CTSR app through mobile technology such
as smart phones and tablets makes access to surgical and
interventional procedures more scalable compared to other VR
simulators. As an open access trainer, platforms such as Touch
surgery have the potential to reach out to trainees on a global scale.
Recently surgical governing bodies are working to highlight the
importance of simulation and rehearsal as an adjunct to surgical
practice. The American College of Surgeons’ strong belief in
simulation has led to the creation of the American College of
Surgeons Accredited Education Institutes (ACS-AEI) with a goal ‘‘to
promote patient safety through the use of simulation’’ [19]. Also,
the role of simulation has been highlighted as an ‘‘important aspect
of training that will increase in significance over the next 5 years’’
by the Trauma and Orthopaedics section of the Royal College of
Surgeons of England (RCSEng). Furthermore, The RCSEng recommended that Touch Surgery and surgical apps could be used to
allow orthopaedic trainees to ‘‘learn operative steps and be
subsequently tested on the move, before (operative rehearsal) and
after surgery’’ [20].
Validation has been demonstrated on numerous other virtual
reality orthopaedic simulators [21–25] however, all these were in
minimally invasive procedures, primarily arthroscopy, and this is
also true for non-orthopaedic simulators [26–30] where the focus
2216
K. Sugand et al. / Injury, Int. J. Care Injured 46 (2015) 2212–2216
is laparoscopy, ureteroscopy or robotics. The current study is the
first to assess the validity of a CTSR for a non-minimally invasive
procedure.
Limitations and future work
One limitation of this study was that no intermediate group was
included. However, we have compensated with adequate numbers
in each cohort and we justified the use of two cohorts for this
validation study. Future work will consist of observing predictive
validity into the operating theatre. Other studies will look at
correlation between performance and visuospatial awareness,
comparator systems, gender, hand-dominance and training effect.
There will also need to be a study in the future on comparing scores
between various VR models to look for transferability of skills; yet
this is difficult to do as similar programmes are limited in number
and not currently available. This will also dictate which
programmes to plausibly implement into formal surgical curricula.
Conclusion
This study demonstrated the construct, content and face
validity of the intramedullary femoral nailing module based on
Touch Surgery, a cognitive task simulation and rehearsal app on
smart devices. Experts significantly outperformed novices in all
four modules but content and face validity were demonstrated in
both cohorts. No such VR platform for assessing this orthopedic
trauma procedure has been done until now. Consequently there
may be a potential to implement Touch Surgery within training
programs to teach and assess orthopedic trainees.
Funding
No funding was provided for this study.
Conflict of interest
None.
Acknowledgements
The authors would like to formally acknowledge the following
people for their help with the study: Ali Nehme Bahsoun, Emre
Doganay and Andre Chow.
References
[1] Weiss RJ, Montgomery SM, Al Dabbagh Z, Jansson KA. National data of
6409 Swedish inpatients with femoral shaft fractures: stable incidence between 1998 and 2004. Injury 2009;40(March (3)):304–8.
[2] Hedlund R, Lindgren U. Epidemiology of diaphyseal femoral fracture. Acta
Orthop Scand 1986;October (57):423–7.
[3] Kneebone R, Aggarwal R. Surgical training using simulation. BMJ 2009;
338(May):b1001.
[4] Department of Health. Guidance on the implementation of the European
Working Time Directive. April 29, 2004. Available at: http://www.dohc.ie/
issues/european_working_time_directive/ [cited 10.01.15].
[5] Fitzgibbons SC, Chen J, Jagsi R, Weinstein D. Long-term follow-up on the
educational impact of ACGME duty hour limits: a pre-post survey study.
Ann Surg 2012;256(December):1108–12.
[6] Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. N Engl J
Med 2006;355(December (25)):2664–9.
[7] Kahol K, Satava RM, Ferrara J, Smith ML. Effect of short-term pretrial practice
on surgical proficiency in simulated environments: a randomized trial of the
preoperative warm-up effect. J Am Coll Surg 2009;208(February (2)):255–68.
[8] Frischknecht AC, Boehler ML, Schwind CJ, Brunsvold ME, Gruppen LD, Brenner
MJ, Hauge LS. How prepared are your interns to take calls? Results of a multiinstitutional study of simulated pages to prepare medical students for surgery
internship. Am J Surg 2014;208(August (2)):307–15.
[9] Promberger R, Ott J, Bures C, Kober F, Freissmuth M, Seemann R, Hermann M.
Can a surgeon predict the risk of postoperative hypoparathyroidism during
thyroid surgery? A prospective study on self-assessment by experts. Am J Surg
2014;208(July (1)):13–20.
[10] Clark RE, Estes F. Cognitive task analysis. Int J Educ Res 1996;25(5):403–17.
[11] Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB,
Jakimowicz JJ, Work Group for Evaluation and Implementation of Simulators
and Skills Training Programmes. Consensus guidelines for validation of virtual
reality surgical simulators. Surg Endosc 2005;19(December (12)):1523–32.
[12] Smink DS, Peyre SE, Soybel DI, Tavakkolizadeh A, Vernon AH, Anastakis DJ.
Utilization of a cognitive task analysis for laparoscopic appendectomy to
identify differentiated intraoperative teaching objectives. Am J Surg
2012;203(April (4)):540–5.
[13] Yates K, Sullivan M, Clark R. Integrated studies on the use of cognitive task
analysis to capture surgical expertise for central venous catheter placement
and open cricothyrotomy. Am J Surg 2012;203(January (1)):76–80.
[14] Campbell J, Tirapelle L, Yates K, Clark R, Inaba K, Green D, Plurad D, Lam L, Tang
A, Cestero R, Sullivan M. The effectiveness of a cognitive task analysis informed
curriculum to increase self-efficacy and improve performance for an open
cricothyrotomy. J Surg Educ 2011;68(September–October (5)):403–7.
[15] Wohldmann EL1, Healy AF, Bourne LE. A mental practice superiority effect:
less retroactive interference and more transfer than physical practice. J Exp
Psychol Learn Mem Cogn 2008;34(July (4)):823–33.
[16] Driskell JE, Copper C, Moran A. Does mental practice enhance performance? J
Appl Psychol 1994;79:481–92.
[17] Arora S, Aggarwal R, Sirimanna P, Moran A, Grantcharov T, Kneebone R,
Sevdalis N, Darzi A. Mental practice enhances surgical technical skills: a
randomized controlled study. Ann Surg 2011;253(February (2)):265–70.
[18] Velmahos GC, Toutouzas KG, Sillin LF, Chan L, Clark RE, Theodorou D, Maupin F.
Cognitive task analysis for teaching technical skills in an inanimate surgical
skills laboratory. Am J Surg 2004;187(January (1)):114–9.
[19] Accredited Education Institutes, American College of Surgeons. Available
from: http://www.facs.org/education/accreditation/aei [cited 10.01.15].
[20] British Orthopaedic Association, Specialist Training in Trauma and Orthopaedics Curriculum August 2014. Available from: http://www.iscp.ac.uk/
documents/syllabus_TO_2014.pdf [cited 11.02.15].
[21] McCarthy AD, Moody L, Waterworth AR, Bickerstaff DR. Passive haptics in a
knee arthroscopy simulator: is it valid for core skills training? Clin Orthop
Relat Res 2006;442(January):13–20.
[22] Muller W, Bockholt U. The virtual reality arthroscopy training simulator. Stud
Health Technol Inform 1998;50:13–9.
[23] Escoto A, Trejos AL, Naish MD, Patel RV, Lebel ME. Force sensing-based
simulator for arthroscopic skills assessment in orthopaedic knee surgery. Stud
Health Technol Inform 2012;173:129–35.
[24] Cannon WD, Eckhoff DG, Garrett Jr WE, Hunter RE, Sweeney HJ. Report of a
group developing a virtual reality simulator for arthroscopic surgery of the
knee joint. Clin Orthop Relat Res 2006;442(January):21–9.
[25] Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK,
Satava RM. Virtual reality training improves operating room performance:
results of a randomized, double-blinded study. Ann Surg 2002;236(October
(4)):458–63 [discussion 463–4].
[26] Ogan K, Jacomides L, Shulman MJ, Roehrborn CG, Cadeddu JA, Pearle MS.
Virtual ureteroscopy predicts ureteroscopic proficiency of medical students on
a cadaver. J Urol 2004;172(August (2)):667–71.
[27] Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, FunchJensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91(February (2)):146–50.
[28] Knoll T, Trojan L, Haecker A, Alken P, Michel MS. Validation of computer-based
training in ureterorenoscopy. BJU Int 2005;95(June (9)):1276–9.
[29] Matsumoto ED, Pace KT, D’A Honey RJ. Virtual reality ureteroscopy simulator
as a valid tool for assessing endourological skills. Int J Urol 2006;13(July
(7)):896–901.
[30] Okamura AM. Haptic feedback in robot-assisted minimally invasive surgery.
Curr Opin Urol 2009;19(January (1)):102–7.