A problem-based learning curriculum and undergraduate performance in the final psychiatry examination at the

November 2013 Vol. 19 No. 4 SAJP 218 In 2001, a problem-based learning (PBL) curriculum replaced the traditional, didactic lecture-based (LB) curriculum at the Nelson R Mandela School of Medicine (NRMSM) in Durban, South Africa (SA). This was prompted by the call from the Health Professions Council of South Africa (HPCSA’s) Medical and Dental Board to modernise curricula at all the country’s medical schools.[1] PBL as a heterogeneous entity has gained wide acceptance across courses and disciplines in higher education.[2] This approach considers both the content and instructional process of a curriculum, and is innovative in that it is a method of teaching as well as a philosophy of learning.[3] Central to the approach is the use of problems, which are posed to students to initiate learning. The approach was conceived at Canada’s McMaster University in 1969, and thereafter adopted at medical schools in Africa either fully or in a hybrid format. Costs for the successful implementation of PBL include library and resource development, refurbishment of small meeting rooms, laboratories for skills training, appointment of educationists and retraining of staff and facilitators. The costly implementation of PBL in the resource-constrained African setting left many clinical teachers wondering whether the reform would be worthwhile.[4] In addition, despite decades of experience with PBL, the debate surrounding its benefits or superiority over traditional methods continues unabated.[5-8] SA has a major shortage of doctors in the public sector and faces concomitant epidemics of HIV and tuberculosis.[9] Due to high patient loads and limited supervision from specialists, it is important for students to be clinically competent on graduation. During their internship they will have to manage illnesses independently and often for prolonged periods under challenging circumstances. There is evidence that students experience PBL environments as more nurturing and enjoyable, and some have argued that this positive learning environment is reason enough for PBL’s adoption.[10] Psychiatry is one of the six major clinical disciplines at NRMSM, and the faculty-wide curriculum revision presented an opportunity to test the effect of the reform on students’ clinical competence in this discipline. We hypothesised that the revision should affect students’ performance positively in the final psychiatry clinical examinations, because the PBL curriculum provides students with (i) earlier clinical exposure to psychiatric patients than LB education and (ii) psychiatric cases that allow the application of basic science knowledge. It also offers (iii) systematic and sequential exposure to clinical presentations, which introduce foundational core principles before more complex presentations; (iv) active learning; and (v) enhancement of communication skills. Curriculum comparison studies that focus on psychiatric knowledge and clinical skills are limited. A study conducted by McParland A problem-based learning curriculum and undergraduate performance in the final psychiatry examination at the Nelson R Mandela School of Medicine

In 2001, a problem-based learning (PBL) curriculum replaced the traditional, didactic lecture-based (LB) curriculum at the Nelson R Mandela School of Medicine (NRMSM) in Durban, South Africa (SA).This was prompted by the call from the Health Professions Council of South Africa (HPCSA's) Medical and Dental Board to modernise curricula at all the country's medical schools. [1]BL as a heterogeneous entity has gained wide acceptance across courses and disciplines in higher education. [2]This approach considers both the content and instructional process of a curriculum, and is innovative in that it is a method of teaching as well as a philosophy of learning. [3]Central to the approach is the use of problems, which are posed to students to initiate learning.
The approach was conceived at Canada's McMaster University in 1969, and thereafter adopted at medical schools in Africa either fully or in a hybrid format.Costs for the successful implementation of PBL include library and resource development, refurbishment of small meeting rooms, laboratories for skills training, appointment of educationists and retraining of staff and facilitators.The costly implementation of PBL in the resource-constrained African setting left many clinical teachers wondering whether the reform would be worthwhile. [4]][8] SA has a major shortage of doctors in the public sector and faces concomitant epidemics of HIV and tuberculosis. [9]Due to high patient loads and limited supervision from specialists, it is important for students to be clinically competent on graduation.During their internship they will have to manage illnesses independently and often for prolonged periods under challenging circumstances.
There is evidence that students experience PBL environments as more nurturing and enjoyable, and some have argued that this positive learning environment is reason enough for PBL's adoption. [10]sychiatry is one of the six major clinical disciplines at NRMSM, and the faculty-wide curriculum revision presented an opportunity to test the effect of the reform on students' clinical competence in this discipline.We hypothesised that the revision should affect students' performance positively in the final psychiatry clinical examinations, because the PBL curriculum provides students with (i) earlier clinical exposure to psychiatric patients than LB education and (ii) psychiatric cases that allow the application of basic science knowledge.It also offers (iii) systematic and sequential exposure to clinical presentations, which introduce foundational core principles before more complex presentations; (iv) active learning; and (v) enhancement of communication skills.
Curriculum comparison studies that focus on psychiatric knowledge and clinical skills are limited.A study conducted by McParland et al. [11] at a British medical school reported that the students from their PBL curriculum achieved statistically significant higher clinical and knowledge-based examination scores than students graduating from the LB curriculum.To date, no published studies have compared the knowledge and clinical skills of graduating students following the revision, in any of the six major clinical disciplines at our school.This study therefore compared the impact of the curricular revision on the psychiatric knowledge and clinical skills of undergraduate final-year medical students in the MB ChB programme at the NRMSM.

Methods
This quantitative assessment audit examined the extent to which the curricular reform affected the assessment outcomes of 10 cohorts of final-year medical students.The psychiatry examination results of five cohorts of students who graduated from the LB curriculum (2001 -2005) were compared with those of five cohorts who graduated from the PBL curriculum (2005 -2009) at the NRMSM.The use of archived student assessment data was approved by the Humanities and Social Sciences Research Ethics Committee of the University of KwaZulu-Natal.

Educational background
Both curricula covered the same content, and students rotated through blocks of equal duration to exactly the same teaching hospitals with the same exit outcomes expected from graduates.There were, however, concerns that the shorter PBL programme would hamper students' ability to achieve clinical competence, and this was the main reason why the examination format remained unchanged.At the end of their degree, the final exit examination for the discipline required students to demonstrate their clinical competence, and consisted of a long case assessment, a written examination which included short questions based on two case vignettes, and an oral examination (viva).

The LB curriculum
Under the LB curriculum, MB ChBVI students spent three years learning pre-clinical outcomes, followed by three years focusing on clinical outcomes.
Students were introduced to the discipline of psychiatry only in their fourth year, and the undergraduate training programme in psychiatry aimed to instill students with the necessary disciplinary content knowledge and clinical skills to identify and treat psychiatric disorders prevalent in the South African primary healthcare and general hospital setting.The teaching method employed a number of didactic lectures.Clinical teaching and training in psychiatry continued through a series of clinical methods modules, offered as a four-week rotation to fifth-year students and as a six-week rotation in their final year.The exit examination occurred at the end of the sixweek rotation in the final year.

The PBL curriculum
The PBL curriculum was five years long (MB ChBV) and has been described in detail elsewhere. [12]The NRMSM adheres to Barrows' [13] core characteristics for PBL curricula: the programme is studentcentred and employs a recognisable sequence of events, where students work on a problem while meeting in small groups under the guidance of a facilitator.The authentic problem serves as a tool to direct students' learning, as they gather new information through self-directed learning strategies and share it with the group in subsequent sessions. [2]Learning was supported by lectures, skills training and clinical exposure, making this a hybrid rather than a pure PBL curriculum.
Psychiatry was introduced in a neuroscience module in the second year of study.Students were thus exposed to disciplinary content much earlier in the PBL curriculum than the LB curriculum.After the second-year module, students were expected to understand the diagnostic classification and clinical presentation of psychiatric disorders, and to be able to perform a mental state examination.As well as basic science, psychiatry was also integrated with other clinical disciplines, e.g.childhood psychiatric disorders were introduced in the 'fever' theme, which focused on growth and development and delirium.There was therefore a major shift from a discipline-based model to a more problem-based model.In keeping with the way in which PBL has been implemented globally, our model emphasised PBL to a greater extent in the pre-clinical than in the clinical years.

Assessing outcomes
The long case examination In the final year of study, a traditional long case examination assesses students' clinical skills.Students are given 90 minutes to take a history, conduct a mental state and physical examination, formulate a differential diagnosis and present a management plan for an untrained patient they have not previously encountered.The examination is uninterrupted and not directly observed.Students then present their findings to two examiners in a structured format.A final mark is based on the consensus of both the examiners.

The case vignette
A 2-hour pen-and-paper examination is used to assess students' diagnostic reasoning skills.They are required to answer short questions with reference to two clinical scenarios.The questions related to each case are marked independently.

The oral examination (viva)
For 15 minutes, two examiners assess students' knowledge on any topic covered in the clinical methods course in psychiatry.The oral examination is primarily a test of factual recall, although students are also expected to answer questions based on simulated case scenarios, demonstrating their ability to apply their knowledge.

Statistical analysis
Curriculum type was the independent variable in this study, and the dependent variables were the results of the long case examination, the case vignette and the viva.Gender, age and matriculation entrance points were the confounding variables.
Mann-Whitney U tests were performed on each of the outcome performance measures and p-values were calculated to show the differences between curricula.The level of significance was p<0.05.
Stepwise multiple linear regression was used to control for possible confounding in the final year.

Demographic characteristics
The current study analysed the psychiatry examination records of 771 students in the final year of the LB curriculum from 2000 to 2005, and compared them with those of 936 students in the final year of the PBL curriculum from 2005 to 2009.
Although there were more female students than males in both curricula, the proportion of female students graduating from the PBL curriculum was greater (Table 1).The mean age for final-year students was higher in the LB curriculum, because it was six years long while the PBL was only five years long.
The results of the multiple linear regression (Table 2 ) showed that students in the PBL curriculum performed significantly better in the problem-solving, case vignette-based examination than LB students (p=0.02).After adjusting for the confounding variables, there were no significant differences in the mean marks for the long case and oral examination between the two groups.

Effect of gender, age and matriculation entrance points on the examination marks
Generally, females, younger students and those who entered medical school with higher matriculation entrance points performed better in both curricula than males, older students and those with lower entrance scores.

Discussion
As assessed by their performance in the long case and oral examination, the psychiatric clinical skills of five cohorts of students emerging from a PBL curriculum were no different from those of five cohorts of students emerging from an LB curriculum.However, the students from the PBL cohort fared better when their ability to solve written case scenarios was assessed.This is hardly surprising, because written case scenarios are the primary trigger for learning in a PBL curriculum.Previous studies examining the effects of PBL on students' problem-solving ability were unequivocally positive. [3,7]owever, although the PBL students were successful problem solvers, this was not generalisable to their clinical reasoning skills as applied to real patients in the long case examination.Clinical reasoning is a complex task and involves much more than the recall of previously learnt information. [3]ur results are in keeping with several other studies which failed to show significant differences in medical competence following the introduction of PBL curricula. [5]Hecker and Violato, [14] utilising United States Medical Licensing Examination performance data from 116 medical schools, showed that various curricular reforms had little effect on students' performance in the examination.
In keeping with the international literature, previous studies showed that our PBL graduates had positive perceptions of their own clinical skills, the PBL learning environment and respect for diversity within the student population, and were generally satisfied with the approach. [15,16]These studies were retrospective evaluations of student and faculty perceptions regarding the curricular revision.Data were collected through surveys or interviews soliciting information from current and former students of the PBL curriculum.The impact of curriculum reform was examined mainly through an evaluation of students' own perceptions of their clinical competencies.Also, none of the studies at our medical school reported on the satisfaction or perceptions of students enrolled in the old LB curriculum.
[7] The positive attributes of PBL were therefore not attained at the expense of clinical competence.
Additionally, the shorter PBL curriculum allowed the medical school to produce an additional 200 graduates compared with what was possible with lecture-based learning (LBL).SA has a low physician-to-population ratio of only 36 doctors per 100 000 in the public sector and there is a need to increase the annual number of medical graduates. [9]y the time the curriculum was restructured at the NRMSM, it was globally recognised that teaching and learning in small groups requires more time, effort and resources.SA has a shortage of both resources and teaching staff.Clinical teaching staff at SA medical schools are mainly public sector specialists, but the number of specialists in this sector has declined. [9]Clinical teaching staff have large case loads and are salaried employees of the Department of Health, for whom patient care is a priority.Clinical teachers also face considerable pressure to publish.The NRMSM Department of Psychiatry has no clinical teachers employed by the university or Department of Education.Innovative pedagogical approaches demand extra time and effort from clinicians, and this must be worthwhile.
Because this study was not an opinion poll and did not rely on student or staff surveys, its findings are not plagued by response fatigue or responder bias.It is also based on a large number of students graduating from one medical school.Furthermore, while many clinical rotations were revised during the nine-year period this study covered, psychiatry was unaffected by the revisions.The method of teaching psychiatry differed significantly between the two curricula but the assessment methods remain unchanged.Identical examination practices make the comparison more robust because there is a perception at the school that the examinations for PBL curricula were more lenient than those for LBL.
However, the study is limited by reporting on the clinical competency in only one discipline within the medical school, and these results may not reflect the overall clinical competencies of the students in the  curriculum reform cohort.Comparative education research based on examination performance is also limited by assessment methods. [7]

Conclusion
Regardless of curriculm, the two cohorts of students displayed the same level of clinical skills in psychiatry.This study did find that PBL had a positive effect on students' ability to solve problems based on written case scenarios, but no striking difference in their knowledge or clinical skills.We therefore conclude that although doctors produced under both curricula possessed similar clinical psychiatric skills, it is not possible to make definitive statements regarding the success or failure of the revision.The expected outcomes of the revision were not clearly specified at the outset, so whether it was worthwhile or not is debatable.
We advocate an outcome-based approach for future curriculum innovations in order to provide a framework for the evaluation of such revisions.This study's contribution to global medical educational discourse is the finding that students' graduating from the PBL curriculum at the NRMSM did not have improved psychiatric clinical skills, but were certainly not worse off than their LBL counterparts.Students trod an alternative path from novice to doctor, but although the road was shorter the outcomes were similar.Our study also emphasises that curricular evaluations are more meaningful when informed by welldefined predetermined outcomes.