Clinical assessment variability - what is really causing it?

There was a recent article in Academic Medicine by Dr. Alexander and colleagues from Brigham and Women's Hospital describing the amount of variability in clerkship grading among US medical schools.  They found that, unsurprisingly, the grading systems for the clinical years had really no consistency at all.  There was inconsistency among the grading systems used (traditional ABCDF or honor/pass/fail or pass/fail) - (table 1), and even within the schools which used a similar scale the percentage of students receiving the highest grade was all over the place (table 2).  So, the question is what do we do with this information?  I think no one really expected findings that were different, but now the answer is out there, in print (or on digital reader screens).

I think part of the answer to where we go from here is to decide if this article was really asking the right question.  The authors do start to talk about this in the discussion section, but I'll try to lay out my thoughts with a little different spin than they gave their discussion.  I think the real question is what are we using the assessment of the clerkship performance for?  What is the essence of what we are trying to measure?  Only when there is broad consensus not only between schools, but within the individual courses of each school will there get to be any semblance of uniformity of grading of students.  I see at least two competing interest which influence how a clerkship director decides to come up with a grading system.  The first is the idea that the students should be measured on how competent they are in the area the clerkship is grading.  In other words, when they are on call as a first-year resident or as a 50 year-old physician, do they have the knowledge and skills to assess a patient with a given problem.  Second, the clerkship director also wants to be sure that the students at their school have a fair chance to compete for selective residency programs.  Thus, there also needs to be a system to distinguish high-achieving from low-achieving students.  The first system is more about the individual student, and with this system, by definition, everyone should be able to achieve the highest score with enough effort and work.  In the second system, it is more about evaluation of the program, and the group.  In this system, it cannot be possible for everyone to achieve the highest score.  However, the system can be manipulated on both sides to aid students or to make it more hazardous.  There are benefits and risks of each system - as with anything in medicine.

I don't think these interests are necessarily incompatible, but they create a tension which I've seen in national meetings and in local curricular meetings.  I also think most clerkship directors are not aware of how this tension affects the grading system they have developed.  I think their not aware as the debates I've heard are usually about tools for assessment or the numbers of honors.  Rarely does the debate get to the level of what is our ultimate purpose for the assessment.  The answer to that question must shape how grades are assessed.  Only when we all become very clear about what we our goals are for the assessment will we truly be able to come to a place where we can have a national dialogue about how to unify the system.

Senioritis in medical school - How to motivate the abulic state

It's that time of year where med students throughout the US shake hands with the Dean, and pull their tassels in unison from the right to the left.  In the months leading up to the tears and endless photo ops that mark any graduation, the students are finishing up the last few rotations of their medical school career.  Although some students retain focus on their swan-song rotations, in every heart there is always the lure of looking beyond the present rotation to the allure of residency in all its glory.  Some have more trouble than others with maintaining drive at the end of the final year.  Most students post-match are taking electives or required courses which are not directly aligned with their chosen field of study.  This makes some sense for getting a Dean's letter together and positioning one's self to be a desirable residency applicant.  Post-match all of this does not seem to matter as much.  Indeed, there has been discussion among academic educators that there is a missed opportunity in the fourth year of medical school largely based on articles like this one from Dr. Lyss-Lerman and colleagues which outlined residency directors view of how well the fourth year is working to prepare for residency.

So, I have a bunch of fourth year students in my neurology clerkship, in fact I have only fourth-year students with the exception of a few third-years that can take neurology as an elective in one block in November.  These students are largely not going into neurology.  How do I try to keep them engaged in our neurology rotation.  (Full disclosure - I'm fully aware I can learn more about how to do this.  I definitely still get some students for whom my little tricks don't work.  This is partly why I'm starting this discussion so that we can all learn from each other).  Here are some of my ideas:

- Have them create their own goals - In orientation, I encourage the students to come up with their own course goals and objectives.  I have prepared goals and objectives, and they are held accountable to those, but there may be specific areas they want to focus on as an area of weakness or as an area which is important for their specialty.  I tell them that in residency, you will not always have clear objectives which are overtly given to you for every rotation.  Thus, I made a habit in residency of picking 2-3 key things I wanted to learn.  When I was on cardiology as an intern, I wanted to sharpen my EKG reading skills and my cardiac exam skills.  Thus, I had something to focus on while taking care of those patients.  Intrinsically created goals are more motivating.  I encourage the students to follow that model as they move on in their career.

-  Encourage exploration of topics related to their field - This is partly a student-led issue, and partly faculty development.  Often students will stay engaged if the faculty recognizes what they are going into, and discusses aspects of a neurological case which is of interest to the student.  For example, we had an OMFS fellow rotating through the neurology clerkship, and I took him aside to discuss a case of a siallorhea I was seeing in the setting of neurodegenerative disease.  Sure it's important for him to know how to treat those diseases from a neurologic standpoint, but he's going to be more interested in the salivary issues.  This can then be used as a doorway to get them to be interested in the rest of the disease.

- Try using games - I haven't used this in my clerkship yet, but as a medical student and a resident, we had an attending (Dr. Harold Adams) who would play Neurojeapordy several times during the rotation.  Students were put into teams, and asked neuro-trivia questions about neuroanatomy, neurological differential diagnosis/treatment, and neurological history.  As a student (and a resident) I really enjoyed this.  It's a way to get students to want to read up on disease states, etc.

- Scare the bejeezers out of them - I will often also play the card that in only two to three short months, they will be responsible for caring for patients on their own (in a supervised fashion initially).  Their signature will mean something, and when someone in their care has a neurological problem, they will likely be the first person to evaluate the situation.  Starting on July 1.  Most students understand this logic.

These are just a few ideas I've used.  Any other thoughts on how to motivate the post-match senior on a required rotation?  Leave them in the comments below!