Rocio Glaria  ( Faculty of Medicine, University of Concepcion, Concepcion, Chile )
Lorena Carmona  ( Faculty of Medicine, University of Concepcion, Concepcion, Chile )
Cristhian Perez  ( Department of Medical Education, Faculty of Medicine, University of Concepcion, Concepcion, Chile )
Paula Parra  ( Department of Medical Education, Faculty of Medicine, University of Concepcion, Concepcion, Chile )

Objective: To relate engagement and academic burnout with curriculum evaluation among speech therapy students.
Methods: This observational, cross-sectional study was conducted at the end of the first academic semester for each level and at the end of a theoretical class in order to ensure the maximum participation rate at the University of Concepción, Concepción, Chile, and comprised students of a speech and language therapy programme.Curriculum evaluation scale, academic engagement and academic burnout questionnaires were used. STATA SE 11 was used for statistical analysis.
Results: Of the 200 participants, 157(78.50%) were women and 43(21.50%) men. The overall mean age was 20.81±2.15 years (range: 18-30 years). Emotional burnout was inversely correlated with the evaluation of teaching and evaluation methods, distribution of fields, teaching team and achievement of objectives (p<0.05 each). Depersonalisation was inversely associated with the distribution of teaching and learning activities (r=-0.13; p<0.05). Lack of personal accomplishment was also significantly and statistically associated with seven out of the eight factors of the curriculum evaluation scale (p<0.05 each). Involvement in studies was statistically and significantly related to all factors (p<0.05 each), except the achievement of objectives. Finally, satisfaction with studies was directly, statistically and significantly related to six of the eight factors (p<0.05 each). 
Conclusion: A more positive evaluation of the academic curriculum was associated with lower levels of burnout and higher levels of academic engagement. 
Keywords: Academic burnout, Academic engagement, Speech and language therapy, Curriculum, Evaluation, University. (JPMA 67: 1362; 2017)

Introduction

Burnout has been initially described as a syndrome that appears in professions that require working with other people, such as health, and derived from the emotional burden and frustrations involved in the social interaction.^1,2 The three dimensions of this syndrome are emotional exhaustion, depersonalisation and lack of personal realisation. These dimensions refer to the wear and exhaustion of emotional resources, a cold attitude of rootlessness and the loss of the ability to contact and finally the lack of competence and professional efficiency related to the personal gains in working with people.^3-6
During the academic period, students must attend to classes, perform evaluations, produce reports and perform practical activities. For this reason, it has been observed that they are also susceptible to burnout. In this case, the condition is denominated academic burnout.^002-4,7,8 In this population the syndrome comprises three dimensions: the emotional exhaustion is understood as the sensation of feeling exhausted against studies, depersonalisation corresponds to an attitude cynical and detached from studies, and finally the loss of personal accomplishment is understood as the perception of being negligent or incompetent with student work.^3,4,7
was created by Schaufeli et al⁹ and validated in 2012 for Chilean university students.^2,7
There are multiple studies that have sought to assess the psychological well-being of students considering burnout as indicator.^10,11 However, this begins form a classical psychological logic focused on health problems, which is opposed to the optics of the positive psychology that emphasises in strengths and protective factors. This, and the conviction that the occupational well-being has led to a construct opposite to burnout, i.e. engagement, is gaining increasing importance.¹²
Engagement has been defined as a positive mental state related to work, lasting over time and not focused on the object or particular situation. From its original conceptualisation, engagement consists of strength, understood as the high levels of energy and mental resilience while working; dedication characterised by a high work involvement along with a sense of pride, enthusiasm and challenge for the work; and the absorption, understood as a high level of concentration and immersion while working, being experienced the notion that "time flies" and having difficulties to interrupt what is being done, because of the strong doses of pleasure and experienced concentration.^5,6
As in the case of burnout, it has also been documented that this state of commitment can be experienced by students, being received the name of academic engagement which is assessed through the Utrecht Work Engagement Scale for Students (UWES-S), created^5-7 and validated by Parra^5,13 for Chilean university students.
Both indicators of psychological well-being of students, i.e. burnout and engagement, would be affected by the academic context in which these students are formed.¹⁴ Especially in health careers, the high degree of stress and psychological distress associated to these formative programmes has been widely documented.¹⁵
On the other hand, recently the training programmes in Chile — following trends of other countries such as Canada or members of the European Union — are being continuously checked in order to bring them closer to competences-based curriculum.^16,17 This change demands modifications in the teaching and learning methods, as well as in the evaluation systems in order to focus the educational process in the student.
Historically, every process of evaluation and analysis of the formative programmes has been performed by those who have found themselves in a position of power, authority or superiority over those who are evaluated. However, Scriven points out the importance of including all people who can collaborate with relevant information regarding the evaluated issues, in order to later produce the improvements that are relevant. From this point of view it becomes crucial to incorporate students as active evaluators of the teaching and learning process.¹⁸ Hence, the evaluation of curriculum could be considered as a process by means of which it is explored about what and how it is taught and learned, and what effect it has for teachers, students and even for society. The task of the evaluator is to reveal the reality of the programme and discard its idyllic versions, so that it is necessary that the evaluator collects information in order to describe realistically what is happening. From this point of view, both teachers and students should perform the evaluation of an educational programme, each contributing with information from their own perspective.¹⁹
Schaufele²⁰ created a questionnaire to evaluate the medical specialisation programme on orthopaedics and traumatology, which was adopted to be applied to other professional training programmes. This questionnaire has been recently validated for medical²¹ and speech and language therapy students.²²
There are currently several investigations on burnout and academic engagement in different populations of students. However, in speech and language therapy the information is scarce. In Chile, this training programme has had a significant growth only since 2003, so that the research during this period has been focused on the subject area rather than in educational issues. Additionally, there is a lack of information about the relation of these phenomena with the perception of the students about the curriculum in which they have enrolled. The current study was planned to co-relate burnout and academic engagement with curriculum evaluation performed by speech and language therapy students of a Chilean traditional university, to count on objective data that allow collaborating in changes of the curriculum in order to ensure a high quality of training.

Subjects and Methods

This observational, cross-sectional study was conducted at the Faculty of Medicine of the University of Concepción, Concepción, Chile, and comprised students of a speech and language therapy programme. Students from first to fifth year were chosen by a non-probability quota sampling. Regular students of speech therapy were included, whereas those who had been absent for more than three months in the preceding year were excluded. We sought to include students from first to fifth year (last year of career) to capture the heterogeneity of the target population and to ensure greater variability of measurements. Those participants who presented more than 10% of missed answers in the instruments were discarded, being obtained a optimal sample size of 200 subjects, with a power of 0.99 (1 - ß = 0.99) and a confidence level of 95% and margin of error of 0.05 (p<0.05). The final sample consisted of the students of each level who assist at the end of a theoretical class carried out by a teacher of the career in order to ensure the maximum participation rate.
Three questionnaires were applied in order to evaluate each of the variables under study, plus a demographic questionnaire designed to collect variables such as gender, age, type of high school from which the person had graduated, having studied a previous career, marital status, children, having studied pre-university courses, and having failed a subject.
To study the academic burnout, the MBI-SS questionnaire was applied.^2-4,7,9 This is composed of 22 statements referred to the feelings experienced with the studies. Students should indicate how often they experienced these feelings, selecting one of the seven alternatives presented to them in a Likert format: never (0), a few times per year (1), once a month or less (2), a few times per month (3), once a week (4), a few times per week (5), and everyday (6). A study by Pérez18 on university students from Chile confirmed that the scale measures the three factors proposed by Maslach and Jackson: depersonalisation and emotional burnout, with possible scores between 0 and 42, and the factor of high personal accomplishment, with scores between 0 and 48. 
Meanwhile, in order to evaluate the academic engagement, the 17-item version of the UWES-S inventory was used.^5,7,9,12 Here, students are also asked about the feelings they have experienced against study, using the same response options of the MBI-SS. According to a study by Parra¹³ carried out with university students from Chile, the inventory measures two factors: involvement in studies, with scores that range between 0 and 72, and satisfaction with studies, with scores between 0 and 30.
Finally, the curriculum evaluation questionnaire, created by Schaufele²⁰ for evaluating the orthopaedics and traumatology, was applied. This questionnaire has 58 items referred to the characteristics of a successful management of a curriculum and students must assess according to the degree in which they agree that these characteristics are present in the study programme. The six possible answers are: strongly disagree (1), disagree (2), indifferent (3), agree (4), strongly agree (5), and no information (0). 
This questionnaire was adapted by Ortiz et al.²¹ to be used in any type of career, being subjected to expert judgment and psychometric evaluation in medical students. Subsequently, its factorial structure was evaluated in speech and language therapy students, outcomes showed that its 58 items were organised in eight factors: relevance of curriculum, teaching and evaluation method, time distribution, objectives of the subject, objectives of the career, distribution of teaching and learning activities, teaching team and accomplishment of objectives.²²
Prior to the completion of this study, an evaluation of the project was conducted by ethics investigation commission of the Faculty of Medicine, to which the career of speech and language therapy belonged. Subsequently, an institutional authorisation with the authorities of the faculty was obtained. Instruments were applied by a professor of the career of speech and language therapy at the end of the first academic semester for each level and at the end of a theoretical class in order to ensure the maximum participation rate. Before applying the questionnaires, students were explained about the objectives of the research, the importance of their participation, the confidentiality and wilfulness of it. Then, students were required to sign an informed consent in which these points were also explained. The questionnaires were self-administered and withdrawn by the researcher when the students completed them. This process took approximately twenty minutes.
As the first step, an assessment of the internal consistency of the scores of the instruments was performed. These instruments were evaluated using the Cronbach\\\'s alpha coefficient. A descriptive analysis of the scores of each factor was performed by using mean, standard deviation, and minimum and maximum values. Then, the correlation of curriculum, engagement and burnout was evaluated by using the Pearson product-moment correlation coefficient based on a unilateral contrast. P<0.05 was considered as statistically significant. Data analysis was performed using STATA SE 11.0 statistical package.

Results

Of the 220 students, 200(90.9%) were selected.Of them, 157(78.50%) were women and 43(21.50%) men. The overall mean age was 20.81±2.15 years (range: 18-30 years). Moreover, 100(50%) participants came from subsidised schools, 69(34.5%) from public schools and 30(15%) from private schools, with one omission. Besides, 48(24%) participants were first-year students, 40(20.00%) second year, 38(19.00%) third year, 42(21.00%) fourth year and 31(15.50%) were fifth-year students, with 1(0.5%) omission. 
As the first step of data analysis, an evaluation of the reliability of the scores of the instruments was conducted by means of the Cronbach\\\'s alpha coefficient, as well as a descriptive analysis of the scores of each factor using the mean. Results show that according to the classification by George and Mallery,²³ six of the eight factors of the Curriculum Evaluation Scale presented a good reliability (Cronbach\\\'s alpha between 0.80 and 0.80), whereas in two the reliability was acceptable (Cronbach\\\'s alpha between 0.70 and 0.90). In the case of the UWES-S, one factor presented an excellent reliability (Cronbach\\\'s alpha higher than 0.90), and the other was acceptable. Finally, in MBI-SSone factor presented a good reliability, and another was acceptable, though in the case of depersonalisation, this presented a questionable reliability (Cronbach\\\'s alpha between 0.50 and 0.60).

.


 Then, by evaluating the correlation of the evaluation that the students made about the curriculum of its undergraduate programme with their levels of burnout and academic engagement, the following results were found.
First, it was found that emotional exhaustion presented significant and inverse association with six of the factors of curriculum evaluation, indicating that who presented greater emotional exhaustion performed a more negative evaluation of the teaching and learning methods (r=-0.19; p<0.01), the time distribution (r=-0.15; p<0.05), objectives of the subjects (r=-0.15; p<0.05), distribution of teaching and learning activities (r=-0.26; p<0.001), teaching team (r=-0.14; p<0.05), and accomplishment of objectives (r=-0.16; p<0.05). 
Meanwhile, depersonalisation was inversely and statistically significantly associated only with the distribution of teaching and learning activities (r=-0.13; p<0.05), indicating that who evaluated this dimension of the curriculum worst were those who presented less personal contact with their studies. In the case of the lack of personal accomplishment, this was inversely correlated and statistically significant with all factors, except in the achievement of objectives (r=-0.01; p=0.86). Similarly, in the case of engagement, the same result was obtained with involvement in studies, in which it was directly and statistically significant with all factors, except the achievement of objectives (r=0.04; p=0.55). On the other hand, the factor satisfaction with studies had a statistically significant direct correlation to six of the eight factors, also with the exception of objective achievement (r=-0.02; p=0.74), being added the evaluation of the teaching team (r=0.09; p=0.20), which was not associated to this factor either
 
.


Discussion

By analysing the relation between academic burnout and the evaluation the students make about curriculum, it was found that the identified relations are reversed, i.e. the more the students experienced some burnout feature, the worse the evaluation of some dimension of the curriculum. This result is similar to what found in the study by Hinrichs et al.¹⁴ on physical therapy students, where it was found that who experienced greater burnout performed a more negative evaluation of the educational environment. This was expected, because as stated by Edelwich and Brodsky (1980), burnout is a process of deception, disillusion or disappointment toward professional tasks, phenomenon that could also occur in students and it explains the worst evaluation of the programme by presenting burnout.²⁴
By performing a more specific analysis by dimensions, it is noted that depersonalisation was the only one that had significant relations with a single factor, the distribution of teaching and learning activities. Therefore, it is possible to assume that having a distant and cynical attitude against study is not an element directly associated with the perception of the students about the educational programme. This is probably due to the fact that this is the dimension with more clinical features of burnout and therefore it would be expected that was more affected by the characteristics of the personality and not only environmental.² This would also explain why in the study by Hinrichs et al.,¹⁴ depersonalisation was the burnout factor that presented weaker correlations with the educational environment. However, the fact that this is associated with the distribution of activities and in turn, this has presented the strongest correlations with the other two dimensions of burnout (r=-0.26), would show that this dimension would be the most critical for students, because a bad distribution could be highly stressful and it could consume the time available for students to recreate, rest and recover from academic activities.
On the other hand, the dimension of emotional exhaustion was not related to relevance of the curriculum and objectives of the career, whereas the lack of personal accomplishment was not related to achievement of objectives. In the case of the emotional exhaustion, the absence of relation with these factors could be due to the fact that these are referred to the more global aspects of curriculum, considering global aspects of the career and not the specific aspects of the subjects with which students have a more direct link. These results agree with those obtained in a study with Colombian psychology students, in which it was shown that students who perceive greater emotional exhaustion reported less satisfaction against studies.²⁵
By assessing the relation between the relation students make about the programme and the academic engagement, direct and statistically significant relations in almost all cases. In the case of involvement, this is expected if the results by Caballero et al.²⁵ are considered, where it was found that students who perceive themselves as more capable show a more positive evaluation of the educational context and give more importance to university, studies, career and academic performance. On the other hand, results presented here are in coincidence with those found in psychology students from the University of Girona, Spain,²⁶ where it was found that satisfaction with studies was positively correlated with the dimensions of engagement, showing that the way in which the curriculum is structured may affect the level of well-being experienced by students.
It should be mentioned that within this analysis the dimensions of involvement with studies and satisfaction with studies are not related to the factor accomplishment of objectives. This could be explained because this factor is referred to the perception on the achievement throughout the training process. For this reason, a more cognitive than emotional process that requires having a global vision of the formative process on the one hand and a clear concept on the objectives of the system on the other. In this case, engagement is referred to an emotional state that although is stable, it would be more related to the direct experience of students.
The current had its limitations as well. It only addressed the phenomenon in a health career and at a particular university, making it necessary to perform similar studies in a broader population considering different universities and training programmes. On the other hand, the burnout, engagement and curriculum assessments were conducted through quantitative techniques, so future studies should complement the information with a qualitative approach which allows sensitivity, thoroughness and depth to the results.
A more favourable evaluation of the academic curriculum was associated to lower levels of burnout and higher levels of academic engagement, showing that the study programme was effectively related to the levels of well-being experienced by students. This pattern of correlations was coincident with that found by Hinrichset al.¹⁴ on Chilean physical therapy students.
However, the factor of burnout depersonalisation was associated only to the factor of curriculum evaluation of distribution of teaching and learning activities. This would indicate that the formative process has a greater association with more dynamic and less clinical aspects of burnout.² On the other hand, in the case of engagement, this directly correlated with almost all evaluation factors of the educational environment, except with that factor that would require a more cognitive and global analysis of the career, suggesting that engagement might be more linked to aspects of immediate experience.  However, it is necessary to take into account that correlations, even though significant, in all cases presented a size of small effect, i.e. it showed weak correlations.²⁷ This shows that despite the studied variables it is highly likely that there are other factors more directly associated, such as the educational environment.¹⁴

Conclusion

A more positive evaluation of the academic curriculum was associated with lower levels of burnout and higher levels of academic engagement. Both phenomena should be considered in every curriculum evaluation process.

Disclaimer: The manuscript was presented at "XV Conference on Education in Health Sciences ", Faculty of Medicine, University of Chile, in May 2015.
Conflict of Interest: None.
Source of Funding: The present work was financed by the Initiation VRID Project N° 214.083.029-1.OIN: "Relation of the engagement and academic levels with the evaluation of programmes conducted by speech therapy students of a traditional university in Concepción".

References

1. Gil-Monte PR, Olivares Faúndez VE.  Psychometric properties of the "Spanish Burnout Inventory" in chilean professionals working to psysical disabled people. Span J Psychol. 2011; 14: 441-51.
2. Pérez C, Parra P, FasceE, OrtizL, Bastías N, BustamanteC. Estructura Factorial y ConfiabilidaddelInventario de Burnout de Maslach en UniversitariosChilenos.Rev Argent ClinPsic. 2012; 21: 255-63.
3. Carlotto M, Pepe A, Goncalves S.  Síndrome de Burnout em estudantes universitários da área. RevistaPsico.2006; 37:57-62.
4. Carlotto M, Goncalvez S. Preditores da Síndrome de Burnout emestudantesuniversitarios.Revista semestral de Associacao Brasileira opsicológia Escolar e Educacion. 2008; 4:101-9.
5.  Parra P, Pérez, C. Propiedadespsicométricas de la escala de compromisoacadémico, UWES-S (versiónabreviada) en estudiantes de psicología. Rev EducCiencSalud. 2010; 7:128-33.
6. Salanova M, Martínez I, BresóE, Llorens S, Grau R. Bienestarpsicológico en estudiantesuniversitarios: Facilitadores y obstaculizadores del desempeñoacadémico. Anales de Psicología.2005; 21:170-80.
7. GlaríaR, Carmona L, Pérez C, Parra P. Burnout y Engagement Académico. Inv Ed Med. 2016; 5:17-23.
8. Gómez P, PérezC, ParraP, OrtizL, MatusO, McCollP, et al. Relación entre el bienestar y el rendimientoacadémico en alumnos de primer año de medicina. Rev. Méd. Chile.2015;143:930-7.
9. Schaufeli W, Salanova M, González-Romá V, Bakker A. The measurement of Engagement and Burnout: A two sample confirmatory factor analytic approach. J Happiness Stud.2002; 3:71-92.
10. Galbraith C, Merrill G. Academic performance and burnout: an efficient frontier analysis of resource use efficiency among employed university students. J Fur Higher Educ.2015;39:255-77.
11. Tomaschewski-Barlem J, Lunardi V, Lunardi G, Barlem E, Silveira R, Vidal D. Burnout syndrome among undergraduate nursing students at a public university. Rev Lat Am Enfermagem. 2014; 22:934-41.
12. Müller R, Pérez C, Ramírez L. Estructura factorial y consistenciainterna de la Utrech Work Engagement Scale (UWES) 17 entre trabajadores de la saludchilenos. Liberabit.2013; 19:163-71.
13. Parra P. 2011. Efectodel Engagement Académicosobre el rendimientoteórico y práctico. PhD diss, Universidad de Concepción, Chile.
14. Hinrichs C, Ortiz L, Pérez C. Relación entre el BienestarAcadémico de Estudiantes de Kinesiología de una Universidad Tradicional de Chile y suPercepción del AmbienteEducacional. Form. Univ. 2016;9:109-16.
15. Freeburn M, SinclairsM.Mental health nursing students\\\' experience of stress: burdened by a heavy load. Int J Ment Health Nurs.2009; 16:335-42.
16. Corvalán O, Hawes G. Aplicación del enfoque de competencias en la construcción curricular de la Universidad de Talca, Chile. Revistaiberoamericana de educación. 2006;40:3.
17. Cisternas M, Rivera S, Sirhan M, Thone N, Valdés C, Pertuzé J, et al. Reforma curricular de la carrera de Medicina de la Pontificia Universidad Católica de Chile. Rev Méd Chile. 2016; 144:102-7.
18. PérezR. La evaluación de programas en el marco de la educación de calidad. Revista de Educación.2002; 21:43-76.
19. Brezmes M. Evaluación Y Optimización de Los ProgramasEducativos. Editorial UOC 2002.
20. Schaufele P. Formación en actitudes y valores en el programa de especialistas en Ortopedia y Traumatología. [tesis]. Concepción: Universidad de Concepción. Facultad de Medicina, Departamento de EducaciónMédica, 2011.
21. Ortiz L, Pérez C, MatusO, ParraP, McCollP, TorresG, et al.Estructura factorial y consistenciainterna de la escala de evaluación de programas en estudiantes de medicinachilenos.Rev. Méd. Chile. (In press)
22. Glaría R, Carmona L, Pérez C, Parra P.Estructura Factorial y ConsistenciaInterna de la Escala de Evaluación del Currículum de ProgramasUniversitarios en Estudiantes de Fonoaudiología de Chile.Rev IberoamDiagnEv. 2016; 41:80-9.
23. George D, Mallery P.  Spss for Windows step by step: A Simple Guide and Reference, 11.0 Update4th ed. Boston: Allyn & Bacon, 2003.
24. CaballeroC, Hederich C, PalacioJ. El burnout acade?mico: delimitacio?ndelsi?ndrome y factoresasociados con suaparicio?n. RevistaLatinoamericana de psicología. 2010; 42:131-146.
25.  CaballeroC, Abello R, Palacio J. Relación del burnout y el rendimientoacadémico con la satisfacciónfrente a los estudios en estudiantesuniversitarios. Avances en PsicologíaLatinoamericana.2007; 25:98-111.
26.  Moral A, Pallisera Y, SantalóE, Gras M, Villar E. Regulaciónmotivacional, satisfacción y rendimientoacadémico de los estudiantes de psicología. Revista de Enseñanza de la Psicología: Teoría y Experiencia.2008; 4:49-65.
27. Cohen J. Statistical power analysis for the behavioral sciences. Routledge:Academic Press, 1988.