Johanna Shapiro, Ph.D.
So Bernice asked me to get the ball rolling in our thinking about an important but thorny issues, evaluation of MH both generally and in residency programs in particular. Working in medical schools we all know that evaluation of learners and of curricula is constant. It is the main tool that faculty use to judge student performance; and the main tool that administration uses to judge out performance!
Of course, we all also know that evaluation is a complex topic. At its core, it involves assessing learners on how well they achieve learning objectives. Evaluation related to the teaching of medical humanities is even more complex because of the difficulty of writing meaningful learning objectives in this area.
When a learning objective involves factual information, it is fairly easy to establish – what is the correct dosage of this drug – but when our learning goals involve values, attitudes, and behavior, it’s trickier.
Measurable outcomes imply things that can be counted, enumerated and listed. While content can be easily measured (who is the author of this poem?; what are 3 ethical principles?), process cannot. Inappropriate quantification can be misleading (is compassion a 4 or a 5?) As one physician scholar once noted, “Not everything that counts can be counted.”
What we want to measure are often intangibles: How our curriculum is intersecting with student values, how does it affect the way they practice in a clinical context, in what sense does it promote critical thinking about patient care, the healthcare system, social justice etc.
There are also instrumental vs. aesthetic arguments, means to an end or an end in itself. One thing that is certainly true at the residency level, and about medical education in general, is the pressure to make humanities and arts education “relevant” to physicians’ daily work (and to some extent) and personal lives. I must admit I have some sympathy for this argument. We are not teaching literature, for example, to undergraduate majors, but to folks who want to become doctors. So connecting the dots doesn’t always seem misplaced, but I recognize there’s plenty to be said on both sides.
Maybe preferable to quantitative evaluation, or at least complementary to it, is narrative self and learner verbal and written evaluation, that indicates trying to determine what the learner got out of the teaching session that matters to their life as a physician. The 360 model is very useful in this regard because it uses continuous circular feedback loops for both teachers and learners. But it is hard to implement in busy residency settings. And it does not solve the problem that most administrators prioritize numerical evaluation, empty as these may be.
So in evaluations we’ve used with residents, they are very much tied in to patient care. When we have a humanities presentation, of course we always have a standard evaluation that includes Likert-type questions such as: To what extent did this presentation 1) encourage you to think differently about patients? 2) increase self and other awareness 3) provide skills for sustaining empathy and compassion in difficult situations etc. However, the following open-ended question seems to attract most attention from residency program directors (PDs): “Describe a past clinical situation where you might have integrated some of the insights/skills you learned today and explain how you could do this.” I think it is the specificity of the responses that interests PDs and makes them feel such sessions are worthwhile. This approach can be modified if an ongoing series is occurring to give learners an assignment of applying one concept/attitude/skill gleaned from a humanities presentation in patient care and then describing what happened.