Shahzaib Habib Soomro  ( Liaquat National Hospital and Medical College, Karachi. )
Syed Sheeraz Ur Rehman  ( Liaquat National Hospital and Medical College, Karachi. )
Farhad Hussain  ( Liaquat National Hospital and Medical College, Karachi. )

Objective: To evaluate the perception of operation theatre educational environment using surgical theatre educational environmental measure.
Methods: This cross-sectional study was conducted at Liaquat National Hospital, Karachi, from August 2015 to February 2016, and comprised surgical and allied trainees. The reliability of the surgical theatre educational environmental measure questionnaire was assessed by Cronbach\\\'s alpha. The minimum score on the questionnaire was 40 and possible maximum score was 200. A score of at least 120 out of 200 was considered favourable. SPSS 22 was used for data analysis.
Results: Of the 103 participants who completed the questionnaire, 52(50.4%) were males and 51(49.5%) females. The results showed favourable operating theatre educational environment with the total score of 129. The overall reliability was calculated to be 0.97. Male residents perceived the educational environment more adequate than females (p<0.05).
Conclusion: The questionnaire was found to be an easy, reliable and practical tool for measuring the perception of operating theatre educational environment.
Keywords: Operating theatre, Educational environment, Learning environment, STEEM, Surgery and Allied. (JPMA 67: 1864; 2017)

Introduction

Medical education has largely escaped from the quality control rigours imposed on clinical practice. ¹ It is considered as the \\\'\\\'soul and spirit of the medical school\\\'\\\'. ² During postgraduate surgical residency courses, most of the training occurs in real clinical environment, i.e. operating theatre. ³ A significant amount of research has been done recently exploring the different aspects of educational environment (EE), which includes the physical, psychological, social and educational domains of a training programme which are thought to play a vital role in the professional and moral developments of the trainees. ⁴ A learning environment (LE) should ideally address all these areas covering the delivery of safety, food and comfort, and also an encouraging atmosphere with constructive feedback. A good LE produces competent healthcare professionals since it influences how, why and what trainees learn. ⁵ The intellectual aspects of surgical training, such as evidence-based practice, learning with patients, and planned teaching should also be addressed. The Liaison Committee on Medical Education (LCME) has stated that medical schools "should regularly evaluate the LE." ⁶ Moreover, the Accreditation Council for Graduate Medical Education (ACGME) has also developed the clinical learning environment review programme as part of their accreditation system. ⁷
The curriculum and students\\\' perception towards EE may affect the quality of learning. ⁸ Historically, EE was considered hard to quantify. ⁹ The evaluation of quality of EE has been demanding and challenging for educators and policymakers. Medical school environment inventory (MSEI) was the first published tool to evaluate the EE in medical education for undergraduates in 1960. ¹⁰ The EE for postgraduates was assessed for the first time in 1993 by Seelig\\\'s residency programme evaluation questionnaire. ¹¹ However, in 1997 Roff et al. developed a practical and valid tool for assessing the educational environment experienced by undergraduate students: the Dundee Ready Education Environment Measure (DREEM). ¹² Following this, there were remarkable developments and tools for measuring LE were developed covering different specialties, including anaesthetic theatre educational environment measure (ATEEM) and general practice. ^13,14 In 2004, Cassar also proposed a 40-item questionnaire called surgical theatre educational environment measure (STEEM).
To equip trainees with a conducive EE, it is vital to have a quality assessment tool. Such an instrument would provide an insight for the quality of an EE in theatre. It will also highlight the problem areas that require rectifications. The STEEM questionnaire has not been previously used in Pakistan to the best of our knowledge. The current study was planned to evaluate the use of the STEEM questionnaire to measure the learning environment in the operating theatre for postgraduate trainees. We focussed on two aspects: do our postgraduate surgical and allied training programmes provide a conducive environment for learning; and are there any differences in perception between male and female trainees?

Subjects and Methods

This cross-sectional study was conducted at Liaquat National Hospital (LNH), Karachi, from August 2015 to February 2016, and comprised surgical and allied trainees. We assessed the practicality of using the STEEM questionnaire by conducting a pilot study in the Department of General Surgery of LNH. The results were promising on a small sample size of 23 with a global reliability of 0.786. This validated the STEEM tool for our EE. A formal consent form was then formulated for the participants including their gender and year of residency. Approval from the institutional ethics and research committee was taken. The STEEM questionnaire contains 40 statements. The participants\\\' responses were calculated using a five-point Likert scale. These ranged from strongly agree (5), agree (4), uncertain (3), disagree (2) to strongly disagree (1). The minimum score was 40 and possible maximum score was 200. A score of at least 120 out of 200 was considered favourable. The value above 120 indicates a more satisfactory educational environment. The questionnaire also asked for information on the gender of the trainees (Table-1).



The STEEM questionnaire was divided into four subscales: trainees\\\' perceptions of their trainer and training (questions 1-13); trainees\\\' perceptions of learning opportunities (questions 14-24); 1. trainees\\\' perceptions of atmosphere in the operating theatre (questions 25-32); and trainees\\\' perceptions of supervision, workload and support (questions 33-40).
The questionnaires were distributed and collected from the participants after completion within a period of 2 weeks. The maximum time taken to fill the each form was 16minutes. We used Education Resources Information Center (ERIC), Google Scholar and PubMed for literature search. The controlled vocabulary (thesaurus and Medical Subject Headings [MeSH]) was merged with key word phrases and terms to narrate LE. These terms were then integrated using Boolean operators.
We included all the surgical and allied trainees working in 12 different departments of LNH. Participants working as medical officers were considered as year-1 residents. We excluded all the registrars and senior registrars from the study to avoid observation bias due to their completion of trainings.
We calculated our sample size by using an online calculator which was found to be 55. ¹⁵ As we included all the surgical and allied trainees working in LNH, our sample size was 104 (Table-2).



We reported descriptive statistics as the minimum, maximum, mean and standard deviation values. The missing values for individual participants were replaced with a score of 3 (the mid-point on the 1-5 scale). The reliability was assessed using Cronbach\\\'s alpha for the whole questionnaire. Mann-Whitney and the one-way analysis of variance (ANOVA) tests were used as non-parametric methods for comparative statistics for assessing gender differences (Table-3).



There were 19 negative statements for which reverse coding were done when they were analysed. We also conducted the factor analysis by using a varimax rotation. Kaiser-Mayer-Olkin (KMO) and Bartlett\\\'s tests were applied. The data was analysed using SPSS 22. Confidence of interval was set to be 95% with 5% margin of error.

Results

Of the 103 participants who completed the questionnaire, 52(50.4%) were males and 51(49.5%) females. Of the 4,120 possible responses, there were only 61(1.5%) missing values. Moreover, 17(16.5%) participants belonged to the general surgery department, 9(8.7%) to neurosurgery, 13(12.6%) to plastics surgery, 14(13.6%) to obstetrics and gynaecology, 15(14.6%) orthopaedics, 4(3.9%) to paediatric surgery, 5(4.8%) to ear, nose and throat (ENT) , 4(3.9%) to maxillofacial surgery, 8(7.8%) ophthalmology, 6(5.8%) to cardiothoracic surgery, 6(5.8%) to urology and 2(1.9%) to vascular surgery.
We calculated the minimum, maximum, mean and standard deviation for each subscale (Table-4).



The mean minimum value of all 40 questions was 2.5 and maximum value was 4.52. The most highly rated statement was, "I get on well with my trainer" (4.47), and lowest rated statement was, "I feel discriminated against in theatre because of my sex" (2.03). There were four statements with a statistically significant difference between genders (p<0.05). Males perceived the educational environment to be more positive than females.

1. Statement 20: The number of emergency procedures is sufficient for me to gain the appropriate experience. (p=0.011)

2. Statement 22: My trainer is in too much of a rush during emergency cases to let me operate. (p=0.043)

3. Statement 24: I have the opportunity to develop the skills required at my stage. (p=0.016)

4. Statement 39: The level of supervision in theatre is adequate for my level. (p=0.011)

The overall global score for the questionnaire given to students was 136 out of 200 (score range 40-200) which is comparable with global scores of Aberdeen study and Birmingham medical students STEEM study (149 and 139, respectively). Considering the four subscales and the overall global score by gender, females scored lower than males on all subscales and in the overall global score. Results of the factor analysis revealed 10 factors, which showed 71% of the variance.
The reliability of STEEM for our study using Cronbach\\\'s alpha scored 0.97 of all 40 statements. Reliability scores for each of the subscales of the STEEM questionnaire were also calculated.

Discussion

As medical education becomes more learner-centred, approaches that shift responsibility for learning become more important. ¹⁶ Residents\\\' evaluation of their EE is a key factor of residency accreditation. It is a strong predictor of resident satisfaction. ¹⁷ A good learning corresponds positively with the trainees\\\' perceptions of the EE, which in turn impacts trainees\\\' learning experiences and outcomes. ¹⁸ Our study appeared to be successful in showing that the STEEM questionnaire is a reliable, dependable and practical tool for evaluating the operating theatre EE of postgraduate surgical trainees in LNH. Overall, the operating theatre EE was perceived to be adequate and acceptable by surgical and allied trainees (136/200). We compared our alpha reliability and mean scores for each of these subscales with that of the original study of basic surgical trainees in Aberdeen and Birmingham medical students STEEM study (Tables-5 and 6).





The reliability was shown to be equal or higher in all subscales in our study compared with Birmingham medical students STEEM study and Aberdeen basic surgical trainees except for the trainees\\\' perception of their trainer and training (Q1-Q13) which had an alpha reliability of 0.72 compared with 0.84 in both the above-mentioned studies.
Our reliability score (Cronbach\\\'s alpha 0.97) was found to be better than the studies done in Aberdeen (Cronbach\\\'s alpha 0.88) and Birmingham (Cronbach\\\'s alpha 0.86). In this study, male trainees perceived the atmosphere in operating theatre more amicable and encouraging than their female colleagues. The race of the trainees was not included in our consent forms as part of the questionnaire. However, addition of trainee\\\'s race in the questionnaire might serve as an important domain for future studies.
We earnestly recommend researchers to use analytic methods to restrict the count of survey items. Factor analysis help researchers to determine the factor structure for EE scores. It can also be of great use to curtail the number of survey items. This will help in reforming the large lists of questions into more easy, quick and short questionnaires. There has already been a study focusing on Mini STEEM with 13 component factor. ¹⁹ In our study, we identified that with the Eigen value set at 1, there were only 10 components factors, which covered 71% of the variance. This will help in modifying the STEEM questionnaires in a more comprehensive form than Mini STEEM in accord to our EE variables.
Any tool used in assessing any EE should be efficient to administer, quick for participants to complete, widely applicable, and sensitive to change over time. ⁴ We found STEEM to be an effective tool for measuring our operating theatre EE.
One limitation of our study was that it was conducted at a single tertiary care set-up; therefore, its findings cannot be generalised. On the contrary, the strengths of our study were its large sample size and the fact that it was the first study of its kind in Pakistan. Also, it was the first time that all the surgical and allied departments have been assessed together using STEEM.
Assessing EE can be very exhaustive. It cannot be sufficiently evaluated by using a single tool alone. There are many factors that should be considered for evaluating any EE. In future we aim to study our EE using a multimodal approach by applying STEEM and Maslach Burnout Inventory together. This will highlight different dimensions for assessing the EE and will create room for further improvement. It may also provide a roadmap to the regulatory bodies concerned for developing legislations in core curriculum and to evaluate resident training programmes. However, further studies and researches are needed to validate these ideas.

Conclusion

The STEEM was found to be an authentic and reliable tool to assess the operating theatre LE for surgical and allied trainees. The overall climate of operating theatre EE was perceived to be adequate with Cronbach\\\'s alpha reliability of 0.97.

Disclaimer: None.
Conflict of Interest: None.
Source of Funding: None.

References

1. Lempp H, Seale C. The hidden curriculum in undergraduate medical education: Qualitative study of medical students\\\' perceptions of teaching. BMJ. 2004; 329: 770-3.
2. Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach. 2001; 23: 337-44.
3. Esteghamati A, Baradaran H, Monajemi A, Khankeh HR, Geranmayeh M. Core components of clinical education: a qualitative study with attending physicians and their residents. Re J Adv Med Educ Prof. 2016; 4: 64-71.
4. Colbert-Getz JM, Kim S, Goode VH, Shochet RB, Wright SM.Assessing Medical Students\\\' and Residents\\\' Perceptions of the Learning Environment: Exploring Validity Evidence for the Interpretation of Scores From Existing Tools. Acad Med. 2014; 89: 1687-93.
5. Bassaw B, Roff S, McAleer S, Roopnarinesingh S, De Lisle J, Teelucksingh S, et al. "Students perspectives on the educational environment", Faculty of Medical Sciences, Trinidad. Med Teach. 2003; 25: 522-6.
6. Liaison Committee on Medical Education. LCME Standards and Publications: Standards.[Online] [Cited 2014 June 4]. Available from URL: https://www.lcme.org/ publications.htm.
7. Accreditation Council for Graduate Medical Education. Clinical Learning Environment Review (CLER) Program. [Online] [Cited 2014 June 4]. Available from URL: http://www.acgme.org/acgmeweb/tabid/436/.
8. Rukban MO, Khalil MS, Zalabani AA. Learning environment in medical schools adoptingdifferent educational strategies. Educ Res Rev. 2010; 5: 126-9.
9. Can STEEM be used to measure the educational environment within the operating theatre for undergraduate medical students? Birmingham Heartlands Hospital, Heart of England NHS Foundation Trust, United Kingdom:Medical Teacher, Vol. 28, 2006; pp-642-7.
10. Hutchins EB. The 1960 medical school graduate: His perception of his faculty, peers, and environment. J Med Educ. 1961; 36: 322-9.
11. Seelig CB. Quantitating qualitative issues in residency training: Development and testing of a scaled program evaluation questionnaire. J Gen Intern Med. 1993;8:610-3.
12. Roff S, McAleer S, Harden RM. Development and validation of the Dundee ready education environment measure (DREEM). Med Teach. 1997; 19: 295-9.
13. Holt MC, Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure(ATEEM). Med Teach. 2004; 26: 553-8.
14. Mulrooney A. Development of an instrument to measure the Practice Vocational Training Environment in Ireland. Med Teach. 2005; 27: 338-42.
15. Creative Research Systems. The Survey Systems. [Online] [Cited 2016 Feb 20]. Available from URL: http://www.surveysystem.com/sscalc.htm.
16. Rotem A, Godwin P, Du J. Learning in hospital settings. Teach Learn Med. 1995; 7: 211-7.
17. Holt KD, Miller RS, Philibert I, Heard JK, Nasca TJ. Residents\\\' perspectives on the learning environment: data from the Accreditation Council forGraduate Medical Education resident survey. Acad Med. 2010; 85: 512-8.
18. Pololi L, Price J. Validation and use of an instrument to measure the learning environment as perceived by medical students. Teach Learn Med. 2000; 12: 201-7.
19. Nagraj S, Wall D, Jones E. The development and validation of themini-surgical theatre educational environment measure. Birmingham Children\\\'s Hospital, Birmingham, United Kingdom, West Midlands Deanery, United Kingdom. Med Teach. 2007; 29: e192-7.