Inam Muhammad Baig  ( Student MSc Clinical Education, University of Edinburgh. Department of Respiratory Medicine, Saint John\'s Hospital Livingston Scotland UK. )

Peer-marked assessment of professional behaviour among medical students has currently been established in most of the developed world. It is one of the standard components of overall summative assessment systems in undergraduate medical education. However, in many places globally, it has not been introduced in final examinations. This paper reviews the recent local studies on professional behaviour in some medical schools of Pakistan and analyses the need of an assessment process. It also examines various studies done throughout the world validating the peer assessment method. Developing on the findings of the local studies and the authentic evidence showing effectiveness of the system internationally, we are proposing a model of peer -marked assessment among medical students in Pakistan. An instrument already found reliable and valid has been adapted in this proposal while at the same time considering local factors in its implementation.
Keywords: Professional behaviour, Peer assessment, Medical student.

Introduction

Professional behaviour in medical practice identifies those observable habits and attitudes of health care professionals which are grounded in the values of humanistic relationships; respect to others, responsibility, honesty, compassion, accountability and seeking academic excellence for benefit of the society.^1-3 Fostering them as an essential component of the total professional competency requires teaching, role modelling and their assessment through available measurement systems starting in medical schools.⁴ Student peer assessment, which means evaluation by students of the same status is one of the methods of assessment in medical education possessing a strong correlation with academic and clinical performance.⁵ It is considered a powerful tool of assessment due to its credible impact on continuous professional development.⁶ This has become particularly relevant and regularly used in the assessment of professional behaviour as a dependable method. In the USA, 50% of medical schools in 2002 were actually measuring professional behaviour in their students through some kind of an assessment system,⁷ which kept increasing and being evaluated.^8-10 However, in many regions of the world, such assessment systems are not in place so far, indicating a strong need of implementing them with identification of local settings.¹¹

Contextual Background

This peer-marked summative assessment of professional behaviours in medical students is designed in context of a developing country like Pakistan where assessment methods of professional competencies do not include assessment of professional behaviour as a separate component in summative examinations. The Pakistan Medical and Dental Council (PMDC) makes it mandatory for all medical colleges (schools) to include teaching of medical ethics in undergraduate curricula¹² and accordingly it is taught during the five years\\\' course.¹³ However, at no stage professional behaviour is assessed, yet being taken among the intended learning goals. This clearly indicates a missing component in a desired equilibrium or constructive alignment, a principle in education, aiming to align the learning activities and assessment systems with the objectives in curriculum and the intended learning outcomes.¹⁴ In Pakistan, information regarding deficiencies in medical professionalism had remained limited to press reports and expert commentaries.^15,16 Few recent observational studies for the first time investigated professional behaviour in medical students.^17,18 and portrayed the incidence of self reported unprofessional behaviour which include plagiarism, cheating, stealing, lying, dishonesty, covering fellow student crimes and bullying etcetera. The substantial issue of academic dishonesty in various forms in university students was also highlighted by Nazir et al.¹⁹ Akhund et al²⁰ and Khan²¹ studied cohorts of medical students in other Pakistani medical schools to find their attitudes towards honesty, respect, dutifulness, accountability and trustworthiness and found that majority strongly believed in all of them as essential needs for a doctor thus conveying a complex discrepancy between ethical beliefs and actual behaviours being demonstrated. To complicate the issue further a study by Sobani et al²² revealed that professional behaviour   largely adhered to similar patterns seen in pre medical school years and instructional efforts in a reputed institution showed no significant improvement. This was the first interventional study with an aim to change the professional behaviour. Although having appropriate methodology but being undertaken in   a highly expensive private medical school, where most of the students were from foreign countries, the results of this study could not be representative of Pakistan and have to be seen cautiously. This evidence of a turmoil in professionalism in Pakistani medical schools necessitates implementation of a reliable and validated assessment system of professional behaviour integrated into the other components of assessment in undergraduate medical education. Peer marked summative assessment would largely contribute to meet these needs.

Purpose

This summative peer assessment of professional behaviours has two main purposes:
1. To ensure that students aim to acquire competency in professional behaviour similar to obtaining competencies in knowledge and skills. This would drive them to transform themselves (if they need so) by doing what is desired and avoiding what is undesired, creating an environment of self regulation. 
2. To screen out those students from progression to further class or to become a doctor, who despite professional behaviour being taught, emphasized and assessed are unable to acquire the minimum standards. This is to safe guard the public interests as an important principle of assessment in medical education.⁵

Peer Assessment as a Summative Assessment System in Medical Education

In addition to their common utility in formative learning to develop and transform, peer assessment systems can reliably be used for summative examinations to take decisions in rewarding qualifications to practice as doctors.²³ There is a general consensus that in view of the high stakes involved in licensure and exit examinations, only those assessment methods should be used for summative purposes which have sufficient psychometric robustness.⁵ Over the last three decades, various peer assessment instruments have proved their reliability and validity due to their use of psychometric characteristics.²⁴ Having this time tested credibility, summative assessments of professional behaviours by peers would be the best model as an essential domain for assessment.²⁵ Peers have been recognized as the most valued source for dependable assessment of professional behaviours based on the facts that they are best informed about the behaviours and interpersonal relationships of their fellows.^26,27

Selection of the Measuring Instrument

In selecting an appropriate tool for summative peer assessment, the criteria to determine the usefulness of an assessment method in medical education described by Van der Vleuten.²⁸ were considered. These include reliability (the capacity of reproducibility), validity (whether the test measures actually what it is meant to measure), educational impact, acceptability and cost effectiveness. A search was done on Ovid collections (uses Medline, PubMed, EBSCO and others) by using key words professional behaviour, medical students and peer assessment. The titles of 172 papers were read and sixteen papers mentioning an instrument were downloaded and analysed. The paper mentioning Peer Assessment Protocol (PAP) by Dannefer et al²⁹6,30-32 thus providing sufficient evidence to its reliability, validity and cost effectiveness. It had also been found acceptable and correlating to effective educational impact in terms of transforming professional behaviours of medical students.⁶

Implementing the Assessment System

The following procedure would be applied to introduce this for the first time in medical schools.

1. The first step would be that a significant number of experts in medical education in Pakistan develop a face and content validity on such an assessment system. Pakistan Medical and Dental Council should then have to agree to this method of professional behaviour assessment as a valid   system to take forward its own code of ethics. Once this authorization is in place, it would be to obligatory for all medical schools both in public and private sector to implement it in their schools.
2. The second step would be that the faculty in every medical school has to become fully aware of this concept of peer assessment of professional behaviours and knows that how it was and is already being practiced in various medical schools all over the world.33-35 In the context of Pakistan, this peer-marked summative assessment will count for 40% of the total assessment score of professional behaviour, the other 60 % would be based on the assessment by doctors, nursing staff and patients.
3. The third step would be to teach medical students. The peer assessment of professional behaviour will be introduced in first year as a teaching module consisting of lectures and tutorials. During the first and second years it would be learnt as a formative assessment method. The first summative peer assessment would be in the last quarter of the third year. The summative peer assessment in fourth year would be the second and the final assessment.
4. This implementation process would pass this design through an initial phase of evaluating its reliability, validity, acceptability and cost effectiveness as the essential criteria of measuring usefulness of an assessment method.

Description of the proposed Assessment Method

This summative peer assessment is designed for 50 medical students in fourth year. To make the assessment more valid, reliable and feasible, two methods would be applied. Firstly, it would be carried out over 10 months\\\' period and every student would be assessed by all of his /her classmates so as to exclude the bias of few. Secondly a similar but different questionnaires would also be completed by doctors, nurses and by patients over the same period and that together would count 60% of the total. We would only include the details of peer assessment relevant to our topic. It was justified in the section of instrument selection that why Dannefer\\\'s Peer Assessment Protocol²⁹ was selected and adapted (Table-1)



as summative peer assessment method in this proposed model. It comprises of 15 Likert items with an additional category of \\\'unable to answer\\\' UA. In our assessment this unable to assess part will be omitted. This Protocol which was later also introduced as Rochester Peer assessment tool in other studies has been mentioned above. Studying and analysing them revealed that all the 15 items together provide evidence about seven core behavioural attributes as shown in Table-2.



These attributes are hidden inside the validated fifteen item instrument of Dannefer at el and include professionally motivated behaviour, academically competitive behaviour, compassionately respectful behaviour, dutifully and sincerely responsible behaviour, intellectual appropriate humility, honesty and interpersonal humanism and trustworthiness. Each month every medical student would be given the task of assessing the professional behaviours of five fellow students being allocated randomly through a system that allocates five new monthly. The names of assessee will be there but assessor would not be identifiable. By assessing five different students every month with four students in the last month, each student would be able to assess 49 of his classmates once in 10 months. Everyone would have 49 peer assessments prepared to make a final summative assessment based on that. All of the 49 assessments on a single student will be used to calculate scores in seven core behavioural attributes of each student as shown in Table-2.
Table-2 is devised for the markers to read peer marked questionnaires and then record scores accordingly. Student would be aware that through these 15 items of the questionnaire what is being assessed however the task of peers would be only to complete the questionnaire. The scores would be calculated and filled in Table-2 by designated staff members. Finally, the scores will be matched to the standard as shown in Table-3 and described with it.



Standards for Assessment

Over all peer summative assessment of professional behaviours would be done from the total 49 individual assessments and not as total marks because each individual student is being assessed once by 49 fellow students.

We would use four statements as standards:

1. Reliable professional behavior
2. No concerns, Compliant student
3. Borderline, Acceptable
4. Serious concerns, Not acceptable

It is shown in Table-3 that if a student scores 60 (all agree) or more (strongly agree) marks in 40 (80%) or more assessments (by 40 peers or more) it would be taken as reliable professional behaviour. If scores 60 or more marks in less than 40 (80%) but more than 30 (60%) assessments it would be considered to have no concerns. If scores less than 60 but more than 44 in 40 or more assessments again no real no concerns. If scores less than 60 but more than 44 in less than 40 but more than 30 assessments would be considered borderline. Any student whose scores do not comply with these four standard statements would be considered to have serious concern in professional behaviour and referred to medical school ethical committee for assessment.

Limitations of this Design

In this design, we have assumed that all students will answer all questions. Through teaching and formative assessments in earlier years and introducing penalties for non-compliance it would be ensured that answering to all questions remains 100%. The second limitation of the design is that in Pakistan individual medical schools would not be able to introduce it as summative assessment system unless approved by Pakistan Medical and Dental Council. This would require an initial presentation of the proposal in a meeting of educational experts to agree on its face validity showing the needs as well as feasibility.  The third potential problem can be resistance from faculties to rely on students for assessment of professional behaviours of their fellows as they may see their own role being given less importance, therefore the peer assessment is being limited to carry only 40% weightage of the total assessment of professional behaviour in the context of Pakistan. The fourth problem with this assessment can be bias (friendly marking). To minimise it, every student would be assessed by at least 50 fellow students. The fifth potential challenge can be total reluctance of PMDC to accept such assessment method of professional behaviour being entirely new in Pakistani context. Its own code of ethics can be used to convince it by experts in addition to portraying authentic evidence showing effectiveness of this method in many regions of the world.

Conclusion

This model for assessing professional behaviours of fellow students as a summative assessment aims to transform current perturbed environment of medical professionalism in Pakistan. The transformation must start from day one in the medical school. By deciding to take this profession, students accepted a task for themselves to acquire enormous amount of new knowledge and skills. They must at the same time recognise and affirm   the task of transforming themselves in their behaviours as per demands of the profession and to uphold the contractual responsibility entrusted on medical profession by the society. This assessment design goals to obtain that kind of educational impact in the long term.

Disclaimer: None.
Conflict of Interest: None.
Funding Sources: None.

References

1.  Epstein RM, Hundert EM. Defining and Assessing Professional Competence. JAMA 2002; 287: 226-35.
2.  Moyer CA, Arnold L, Quaintance J, Braddock C, Spickard A, Wilson D, et al . What factors create a humanistic doctor? A nationwide survey of fourth-year medical students. Acad Med 2010; 85: 1800-7.
3.  Cameron RA, Mazer BL, DeLuca JM, Mohile SG, Epstein RM. In search of compassion: a new taxonomy of compassionate physician behaviours. Health Expect 2015; 18: 1672-85.
4.  Stern D T and Papadakis M. Medical Education: The Developing Physician — Becoming a Professional. N Engl J Med 2006; 355: 1794-9.
5.  Epstein RM. Assessment in Medical Education. N Engl J Med 2007; 356: 387-96.
6.  Nofziger AC, Naumburg EH, Davis BJ, Mooney CJ, Epstein RM. Impact of peer assessment on the professional development of medical students: a qualitative study. Acad Med 2010; 85: 140-7. 7.  Arnold L. Assessing Professional Behaviour: Yesterday, Today, and Tomorrow. Acad Med 2002; 77: 502-15. 
8.  Mazor K, Clauser BE, Holtman M, Margolis M J. Evaluation of Missing Data in an Assessment of Professional Behaviours. Acad Med 2007; 82: S44-47.
9.  Parker M, M Litt, Luke H, Zhang J, Wilkinson D, Peterson R et al. The "Pyramid of Professionalism": Seven Years of Experience With an Integrated Program of Teaching, Developing, and Assessing Professionalism Among Medical Students. Acad Med 2008; 83: 733-41.
10.  Ziring D, Danoff D, Grosseman S, Lange D; Esposito, Amanda MS et al. How Do Medical Schools Identify and Remediate Professionalism Lapses in Medical Students? A Study of U.S. and Canadian Medical Schools. Acad Med 2015; 90: 913-20.
11. Khan JS, Tabasum S. Medical professionalism: a panoramic view through the kaleidoscope of stakeholder perspectives. J Ayub Med Coll Abbottabad 2011; 23: 138-44.
12.  Pakistan Medical and Dental Council: Code of Ethics. 2011. Available at www.pmdc.org.pk/ethics/tabib/101/default. Downloaded on 22 Jan 2017.
13.  Butt NH,   Khan JS. Medical professionalism - validity of arabian "lamps" in the context of medical education in Pakistan. Biomedica 2015;   31: 46-52.
14.  Biggs J. Enhancing Teaching through Constructive Alignment. High Educ 1996; 32: 347-64.
15.  Gadit AA . Professional behaviour of doctors: are we meeting the criteria? J Pak Med Assoc 2007; 57: 425-7.
16.  Shiwani MH, Gadit AA. Medical negligence: A growing problem in Pakistan, J Pak Med Assoc 2011; 61: 610-1.
17.  Ghias K, Lakho GR, Asim H, Azam SA , Saeed   SA. Self-reported attitudes and behaviours of medical students in Pakistan regarding academic misconduct: a cross-sectional study BMC Med Ethics 2014; 15: 43.
18. Shukr I, Roff S. Prevalence of lapses in academic integrity in two Pakistani medical colleges, Med Teach 2015; 38: 470-5.
19.  Nazir MS, Aslam MS. Academic dishonesty and perceptions of Pakistani students. Int J Educ Manag 2010; 24: 655-68.
20.  Akhund S, Shaikh ZA, Ali SA. Attitudes of Pakistani and Pakistani heritage medical students regarding professionalism at a medical college in Karachi, Pakistan. BMC Res Notes 2014; 7: 150.
21.  Khan AZ. Non-academic attributes of hidden curriculum in medical schools. J Coll Physicians Surg Pak 2013; 23: 5-9.
22.  Sobani ZU, Mohyuddin MM, Farooq F, Qaiser KN, Gani F,Bham NS et al. Professionalism in medical students at a private medical college in Karachi, Pakistan. J Pak Med Assoc 2013; 63: 935-9.
23.  Steverding D, Tyler KM , Sexton DW. Evaluation of marking of peer marking in oral presentation Perspect Med Educ. 2016; 5: 103-7.
24.  Speyer R, Pilz W, Kruis JVD, Brunings JW.. Reliability and validity of student peer assessment in medical education: A systematic review. Med Teach 2011; 33: e572-e585.
25.  Schönrock-Adema J, Heijne-Penninga M, Van Duijn MAJ, Geertsma J, Cohen-Schotanus J. Assessment of professional behaviour in undergraduate medical education: peer assessment enhances performance. Med Educ 2007; 41: 836-42.
26.  Arnold L, Shue CJ, Kalishman S, Prislin M, Pohl C, Pohl H, et al. Can there be a single system for peer assessment of professionalism among medical students? A multi-institutional study. Acad Med 2007; 82: 578-86.
27.  Shue CK; Arnold L; Stern DT. Maximizing participation in peer assessment of professionalism: the students speak. Acad Med 2005; 80: S1-5.
28.  Van der Vleuten CPM. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract 1996; 1: 41-67.
29.  Dannefer EF, Henson LC , Bierer SB, Grady-Weliky TA, Meldrum S, Nofziger AC, et al . Peer assessment of professional competence. Med Educ 2005; 39: 713-22.
30.  Epstein RM, Dannefer EF, Nofziger AC, Hansen JT, Schultz SH, Jospe N, et al. Comprehensive assessment of professional competence: the Rochester experiment. Teach Learn Med 2004; 16:186-96.
31.  Lurie SJ, Nofziger AC, Meldrum S, Mooney C, Epstein RM. Temporal and group related trends in peer assessment amongst medical students. Med Educ 2006; 40: 840-7.
32.  Donnon T, McIlwrick J, Woloschuk W. Investigating Reliability and Validity of Self and Peer Assessment to Measure Medical Students Professional Competencies. Creat Educ 2013; 4: 23-8.
33.  Ginsburg S, Regehr G, Stern D, Lingard L. The Anatomy of the Professional Lapse: Bridging the Gap between Traditional Frameworks and Students\\\' Perceptions. Acad Med 2002; 77: 516-22.
34.  Garner J, Mckendree J, O\\\'Sullivan H and Taylor D. Undergraduate medical student attitudes to the peer assessment of professional behaviours in two medical schools. Educ Prim Care 2010; 21: 32-7.
35.  Guraya SY, Guraya SS,   Mahabbat NA, Fallatah KY, Al-Ahmadi BA, Alalawi HH. The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine. J Clin Diagn Res 2016; 10: JE01-JE05.