Student selection for Undergraduate Medical Education Programmes (UGME) are high stake decisions with considerable economic implications for faculty, institution, society and students and usually, institutions have a large pool of applicants with good academic records to select from. ^1,2 Health care practices, however, requires many other attributes like communication skills, compassion, ability to maintain dignity and respect of patient, professionalism, critical thinking and problem solving in addition to cognitive abilities. ^3,4 The recent move to outcomes or competency-based curricula is motivated by the recognition that the presence of these attributes is required for a holistic and better quality of patient care. ⁵
In most countries assessment of students\\\' cognitive abilities include evidence of prior academic achievement, such as undergraduate grade point averages (UGPA) and Medical College Admission Test (MCAT) scores in USA, Canada and Australia and A\\\' levels, General Certificate of Secondary Education (GCSE) in UK. ^1,6,7 These selection tools do not, however, assess students\\\' non-cognitive attributes which are so essential for their future practice as health care professionals. ^8,9 The traditional interviews used worldwide for this purpose are beset with reliability and validity issues in addition to incurring huge costs. ^1,3,9,10 With evidence emerging about the association between the non-cognitive admission criteria and better performance in medical school, post graduate training and clinical practice, it is essential to use admission methods which assess these non cognitive attributes in a valid and reliable manner. ¹¹
Eva et al developed the multiple mini-interviews in 2004 for assessment of multiple non-cognitive constructs. Utilizing the structured multiple sampling approach used in Objective structured Clinical Examination (OSCE), scores on MMI have shown to be reliable and valid for assessing personal traits and exhibit generalizability to clinical and licensing examination performance. ¹²
Since the regulatory body, Pakistan Medical and Dental Council (PMDC), has not included interviews in admission guidelines, only two Pakistani medical colleges including the Aga Khan University are using interviews for assessment of non-cognitive domain. ¹³ To date no study has been reported on MMI for admissions into medical colleges in Pakistan. This study reports the development and administration of MMI in a private medical college in Pakistan. It assesses the descriptive and psychometric properties of the MMI station scores used and the construct validity of MMI stations.

Methodology

MMI Station Development: After getting permission from the Institutional Review Board, a core team developed a blueprint comprising of nine important attributes aligned to the institutional vision as well as the identified curricular outcomes after extensive literature search. ^3,7,11,12 Nine attributes identified by the core team were critical thinking, problem solving, and communication skills, working in a health care system, cultural sensitivity/social awareness, ethics, honesty/ integrity and  punctuality. Each attribute was operationally defined and one scenario for each of them was constructed. These 5-8 line scenarios served as a trigger and were followed by a question: "What will you do in this situation?" Each station had 3-5 items testing the underlying attribute. A 5 point rating scale was used to rate each item on the station and then converted into percentage cores for analysis and reporting purposes.
MMI Administration: The MMIs were administered to a total of 365 students over 6 days with 6-7 sessions held each day. Each applicant was assessed by 9 different assessors on nine different MMI stations.
Descriptive and Psychometric Analysis of MMI Stations: In addition to the mean scores, standard deviation and number of items for each station, the reliability coefficient using Cronbach\\\'s alpha, standard error of measurement and item-total correlation of the scores on each station under study were determined.
Factor Analysis for Construct Validity: To determine the number of factors being assessed on MMI examination as a whole and to assess the construct validity of each station we used item-wise data for all stations for Exploratory Factor Analysis. We analyzed this data using principal component analysis, with varimax rotation following Kaiser Rule (i.e. eigenvalues > 1.0).

Results

The number of items, descriptive and psychometric properties of the nine MMI stations is shown in Table-1.



The mean scores on each MMI station ranged from 27.4% to 80.0%. The reliability of these MMI stations using Cronbach\\\'s alpha ranged from 0.64 to 0.98, and the standard error of measurement from 3.41% to 8.97%. The item-total correlations ranged from 0.53 to 0.96 except for one item on the station on empathy which had an item total correlation of 0.24.
The results of the factor analysis for the 37 items on the nine MMI stations are shown in Figure and Tables-2. As shown in Figure and Table-2, the data loaded on a total of ten factors converging after six iterations following the criterion of eigenvalues >1.0. These ten factors accounted for 78.3% of the total variance (Table-2).

Discussion

To be a successful medical student and ultimately an effective and competent doctor, a prospective student needs to possess a range of non-cognitive skills, qualities and positive attitudes along with the academic ability. ^7,14 Our results show good reliability and construct validity of the MMI stations developed for selecting undergraduate students on basis of non-cognitive attributes essential for their future performance as professionals in a resource constrained environment.
Eva et al¹² in their seminal work had focused on assessing communication skills, punctuality, critical thinking and problem solving and working in health care systems which were consistent with our institutional philosophy and outcomes as well. In our study we have assessed five additional attributes mainly cultural sensitivity, empathy, punctuality, responsibility and reliability which were also studied by Lemay et al. ¹¹
The reliability of our MMI stations using Cronbach\\\'s alpha ranged from 0.64 to 0.98, which is similar to the study by Lemay et al. ¹¹The high Cronbach\\\'s alpha scores in our study provide evidence of high item cohesiveness among the subscales of each station as well as the evidence of stable scores for each applicant. ^10,15 Our analysis shows that our students scored well on stations testing critical thinking, problem-solving, communication skills, working in health care systems, honesty, responsibility and reliability as shown by the mean score of the stations. The stations showed high internal consistency and item-total correlations validating that the content of each station was assessing the desired attribute.
The students\\\' performance on the station assessing cultural sensitivity/social awareness was, however, poor. Culture influences not only health practices but also how the healthcare provider and the patient perceive illness. Health care providers need to be culturally competent so that they are more compassionate and caring to the needs of the people they serve. ¹⁶ The reasons for the poor performance of students on this station could be due to assessor\\\'s bias or lack of training, and lack of exposure of our students to such issues during high school education. In our study we assessed our students on communication skills and empathetic approach to patients as these are considered to be the mainstay of medical care and good communication skills are highly correlated with better patient adherence. ^16,17
So far the evidence of using MMI in undergraduate student admission process has come from resource rich countries like USA, Canada, Australia, Saudi Arabia and Israel. ^1,9,17 Though the costs of conducting MMI are more than written exams, they are considerably less than traditional interviews and MMI are more time efficient as well. ^18,19 However the significant additional information they provide about the non cognitive attributes as well as the evidence of their ability to predict future performance, calls for a wider usage. The only study from Pakistan using MMI for resident selection comes from AKU. The study also cites resource intensiveness of MMI as compared to interviews as the reason for its limited sample size of 16 residents and eight stations. ²⁰ In our study, we were able to develop and train faculty with minimum expenditure and within a very short span of time which augers well for private as well as public medical colleges of a developing country.

Conclusion

Our study shows that MMI can be developed and implemented within restricted resources and provides evidence for other institutions for adopting this method of student selection in place of the traditional interviews. The descriptive analysis was reported to provide psychometric evidence. The reliability and SEM of our MMI were found to be acceptable for most of the stations. Factor analysis revealed that the stations assessed the attributes that were intended to be assessed. The main purpose of using MMI is to make reliable and valid decisions to select students with desired non cognitive attributes.

Acknowledgement

Special thanks to Prof. Dr. Mobeen Iqbal; who gave us a critical review on the project and provided us scientific advice.

References

1. Dowell J, Lynch B, Till H, Kumwenda B, Husbands A. The multiple mini-interviews in the U.K. context: 3 years of experience at Dundee. Med Teach 2012; 34: 297-304.
2. Leinster S. Selecting the right medical student. BMC Med 2013; 11: 245.
3. Eva KW, Reiter HI, Rosenfeld J, Trinh K, Wood TJ, Norman GR. Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. JAMA 2012; 308: 2233-40.
4. Oluwasanjo A, Wasser T, Alweis R. Correlation between MMI performance and OSCE performance- A pilot study. J Community Hosp Intern Med Perspect 2015; 5: 10.
5. Roberts C, Clark T, Burgess A, Frommer M, Grant M, Mossman K. The validity of a behavioral multiple-mini-interview within an assessment centre for selection into specialty training. BMC Med Educ 2014; 14: 1-11.
6. Ali A, Ali Z. Admission policy of medical colleges: evaluating validity of admission test in Khyber Pakhtunkhwa, Pakistan. J Res Reflect Educ 2013; 7: 77-88.
7. Koenig TW, Parrish SK, Terregino CA, Williams JP, Dunleavy DM, Volsch JM. Core personal competencies important to entering students\\\' success in medical school: what are they and how could they be assessed early in the admission process? Acad Med 2013; 88: 603-13.
8. Pau A, Chen YS, Lee VK, Sow CF, De Alwis R. What does the multiple mini interview have to offer over the panel interview? Med Educ Online 2016; 21: 10
9. El Says F, Ayuob N, Fahmy AR, El Fayez F, Hasanian M, El Deek B. Experience of establishment of multiple mini structure interviews as part of student admission policy at Faculty of Medicine, King Abdulaziz University. Med Teach 2013; 35(Suppl 1): S74-S7.
10. Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo1vishna C, Nadarajah D. The Multiple Mini-Interview (MMI) for student selection in health professions training - A systematic review. Med Teach 2013; 35: 1027-41.
11. Lemay JF, Lockyer JM, Collin VT, Brownell KW. Assessment of non-cognitive traits through the admissions multiple mini-interview. Med Educ 2007; 41: 573-9.
12. Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ 2004; 38: 314-26.
13. Rahbar MH, Vellani C, Sajan F, Zaidi, AA, Akbarali L. Predictability of medical students\\\' performance at the Aga Khan University from admission test scores, interview ratings and system of education. Med Educ 2001; 35: 374-80.
14. Kirch DG. Transforming admissions: The gateway to medicine. JAMA 2012; 308: 2250-1.
15. Albanese MA, Snow MH, Skochelak SE, Hugget KN, Farrell PM. Assessing personal qualities in medical school admissions. Acad Med 2003; 78: 313-21.
16. Tiller D, O\\\'Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Med Educ 2013; 47:801-10.
17. Peskun C, Detsky A, Shandling M. Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Med Educ 2007; 41: 57-64.
18.          Hissbach JC, Sehner S, Harendza S, Hampeb W. Cutting costs of multiple mini-interviews - changes in reliability and efficiency of the Hamburg medical school admission test between two applications. BMC Med Educ 2014; 14: 54.
19. Rosenfeld JM, Reiter HI, Trinh K, Eva KW. A cost efficiency comparison between the multiple mini-interview and traditional admissions interviews. Adv Health Sci Educ 2008; 13: 43-58.
20. Andrades M, Bhanji S, Kausar S, Majeed F, Pinjani S. Multiple Mini-Interviews (MMI) and Semistructured Interviews for the Selection of Family Medicine Residents: A Comparative Analysis. International Scholarly Research Notices 2014; 2014: Article ID 747168.