Using a Lego to explain the difference between competencies and EPA's

People in medical education often have trouble figuring out the difference between competencies and EPA (entrustable professional activities). There is a pretty big philosophical difference. The competencies are definitions of observable behaviors and the EPA's are about observing a learner do a specific work task. Here is a recent article from Carraccio and others  that tries to ties the concepts together.

I was in a meeting yesterday where we were discussing the differences between EPA's and competencies. The group was trying to determine whether you are obligated to assess one first. We have 43 competencies in our new curriculum and 13 EPA's. The question that came up was if EPA 2 is going to be assessed in a student, and it is identified to require multiple competencies, do I need to measure the competencies first to allow me to get into an assessment for EPA's? The reverse of this question is if I am found to be entrustable to an acceptable level for graduation for EPA2, does this automatically allow me to be entrustable on all the related competencies.

While this discussion was going on, my mind wandered to Legos. I've been building Lego sets for years. My son and daughters now have large tubs of Legos in our house. It's really cool how you can make all sorts of wonderful things with the simple building blocks that are Legos. You can think of competencies as the individual building blocks. These are the behaviors necessary to build cool stuff. If you don't have the basic building blocks, you can't really make many cool sets (EPA's). The blocks come in lots of different shapes. Think of each shape as a competency. There are long flat short pieces and long flat long pieces. There are two by four bricks and two by eight bricks. There are all sorts of bricks. The bricks also come in different colors which can represent that a competency must be demonstrated in many different environments prior to saying for sure that it has been acheived. In other word, you may be good at applying medical knowledge in a pediatrics outpatient clinic, but not in an inpatient ICU with a critically ill patient. So, to check out on any given competency, the student may need a green two by four brick (applying medical knowledge in peds clinic), and a red two by four brick (applying medical knowledge in an ICU).

EPA's then are like the ability to build the sets. An EPA would be like taking all those Lego bricks and putting them together to make a car or a boat or a house. The act of making the car or boat or house means that you not only have the bricks needed to make the set, you can use them appropriately. So to enter an order in the ICU would be like making a house. The learner needs some two by four red bricks to make the house, but will also need roof pieces (say an infomatics competency) as well as other pieces. And they need to all be the right color to make a house in the ICU setting. Having a red two by four brick does not mean a student can build a house (they need specific skills to put it all together and other pieces), and building a house in the ICU does not mean you can build a house in the peds clinic (you need green pieces for that).

So, in other words, the EPA's and competencies are each dependent on each other. But both need to be assessed in parallel to assure that students Lego buckets are full of lots of cool and useful pieces, but also to assure that they can actually use the cool pieces to make stuff. Let me know if this helps you understand how EPA's and competencies work together, and what you think of this analogy in the comments below.

In which I discuss educational philosophy heresy

Educational reform always provokes controversy and arguments. I honestly started engaging in healthy discussions (read arguments) about educational reform when I was in my first education theory class in college. Most education reform arguments come down to whether the current paradigm is really broken, and whether the new paradigm is enough better to be worth the trouble replacing the old paradigm. Most of these arguments have fuzzy data at best to show for either side of the argument.

In my experience, these arguments tend to ride heavily on the past educational experiences of those involved in the arguments. The problem with this approach is that it assumes the two (or three or four) argue-ers are all equivalent learners. It assumes that all learners will thrive in the environment in which the argue-er thrived. It assumes that all learners have a mental processing intake system which acquires and stores information in a similar manner. It assumes that all learners are motivated to learn by the same motivations that drove the argue-er. No wonder these arguments are typically never resolved with the one party spontaneously saying, "Wow, you're right, and I was wrong all along. Thank you!".

Why is this? I think it is because we often make the assumption that all learners are equivalent in every aspect of acquiring, storing, retrieving, and applying knowledge. This makes it easier for us to create what little data we have as our current model for getting data on effectiveness of educational models. A p-value is not so useful if the entire cohort you are studying is a ill-defined mass of goo. Unfortunately, that is exactly what we have as our substrate, an ill-defined mass of goo.

What do I mean by this? Take neuroscience education in medical school as an example. First, there are obvious background differences - people with advanced degrees in neuroscience mingle in the class with those who have no idea what the frontal lobe is all about. Second, the way people learn is different. When I was a resident, I liked to see a few patients, and then take time right then to look up a bunch of stuff about those patients. I had friends who would rather be slammed with as many patients in a shift as they could find, as they felt they learned better in the doing. Some people like learning large concepts, and then going into details, and others like learning the details first, and then piecing them together later into a larger whole. Some people like to focus on one system or organ at a time, and some people like to have multiple courses concurrently running, so there is more time to absorb the information from each course. Some people love concept maps. Personally, I've never been able to get my head around why they are so great. I'm more of an outline guy. With these differences, we are trying to measure and argue over substrate that is an ill-defined mass of goo.

I'm not saying there are not basic learning theory principles which can be universal. I am saying the application of those basic learning theories is sometimes more wibbly wobbly than the ed-heads like to let on in their arguments. It could be that this multiple choice test on whether education reform is needed is not really a multiple choice test. It's an essay test. And there are multiple right answers as long as you can justify your answer. And everybody hated those tests...

Is there an upside to the noise of seemingly irrelevant content in medical education?

As I have worked in planning curriculum in medical school either as a course/ clerkship director or in various school-level committees, a common question keeps coming up. How much is too much information?  Or conversely how little do you need to know about basic science to be able to competently practice medicine?

The age of the 'Google' search and other tools like Epocrates and PubMed being capable of getting a seemingly endless stream of factoids answered instantly makes this question more difficult. How much do I need to know versus how much do I need to be able to know how to look it up?

I don't think there is a perfect answer to this question, but let's use as an example GI histology. As a neurologist, if you asked me how much I have used my knowledge from medical school of GI histology in the past six months, I would probably laugh. Most neurologists would probably laugh as when you ask that, what comes to mind immediately is the day in med school microscopy lab where we looked at the slides of intestines and identified the villi cells. I don't do that anymore - like ever.

However, in the last six months I have taken care of Parkinson's patients with gut motility problems and constipation and I've also taken care of people on whom we were considering gluten-sensitivity as differential diagnostic points. We also know the carbidopa/levodopa competes with protein in small intestine absorption. How much of my ability to understand these basic problems with occur daily in my clinic is founded in part on my original knowledge of GI histology? What I think it critical to consider when considering what level of detail of GI histology is important to physician training is to consider what is implicit knowledge that allows me to solve problems. This means looking beyond the typical response to any given topic where a practicing physician says, "I never use that." (Biochemistry anyone?) This means spending time unpacking the implicit framework knowledge on which you have built much more complex concepts. On the flip side, there are some things which I learned in med school which I really don't ever seem to use much now even as much as I try to rack my brain to figure out if I do use them.

I'm not sure of the best way to puzzle this question out. I'm a little worried about running the grand experiment of just stopping teaching the med students all the tiny details we have taught in the past without first pausing to understand the repercussions. There is not likely going to be a firm line in the sand somewhere where a given topic is relevant or irrelevant, it'll look likely more like a large sandy smudge. However, every teacher of med students has to draw their line somewhere, and it would be good to have some alignment within a med school system.

How might a pure competency-based curriculum change residency interview season?

OHSU is one of several schools that recently received an AMA-funded grant to push medical educational innovation.  Our new curriculum, YourMD (yeah it has a cool marketable name), is in many ways a test lab for this grant (to be clear, most of what I'm going to discuss here is beyond the scope of the current version being developed for the YourMD curriculum, and I'm outlining my personal view of what the model may look like in the future).  One of the primary themes in OHSU's work for this grant is to create a workable competency-based (not time-based) model of medical education.

Multnomah County Hospital residents and interns, circa 1925

 

 

As you can imagine, there have been many questions about the logistical problems with such a system.  One of the issues raised at our institution as this concept has been discussed at various faculty meetings is the perceived trouble students in such a system will have in finding a residency program.  After all, the student will have this transcript which looks remarkably different from most of the current school transcripts.  It will have a bunch of competencies and EPA's.  It may not have any mention of honors.  How is a residency director to be able to choose who is the best candidate for their program?

I've thought about this a bit, and have a few ideas.  First, if the school is truly competency-based, just the fact that the student has been able to graduate should indicate that:

a) The student understands and applies the knowledge necessary to start as an intern,
b) The student clearly demonstrates the skills necessary to start as an intern
c)  The student clearly demonstrates the professionalism necessary to start as an intern
 
To my mind (assuming the system will work as advertised), this is revolutionary.   This means you don't have to guess as a residency director what you are getting.  You don't have to read between the lines for the secret codes hidden in the letters of recommendation.  This person is ready for residency
.  End of line.

So, then what do you look for now?  Now, as a program director, you can begin to look more at what other experiences and skills does this particular individual have that would help them thrive at any particular institution. Instead of trying to assure that the person had 'honors' in internal medicine, the medicine program director can sort applicants in all manner of ways. They could determine their program wants people who have above-average skill in quality-improvement, or they could decide they want residents who are particularly interested in medical education. They can rank based on how well they operate in a team-environment. They can look for students who have had particular experiences that would benefit them in their environment - say a lot of rural practice experience or many rotations in an under-served inner-city.  Each program director can choose what they'd like to highlight, and I don't see a problem with letting students know what they are looking for in applicants. This makes the interview sessions even less about figuring out if this person can operate on the ward successfully, and more about does this person fit well with our system and our culture.

If competency-based education works, this may be something residency program directors will need to think about. We're all well on the way to competency-based education. So, program directors, prepare yourselves. I think it'll make interview season more fun actually.

6 things to make your medical school lectures better

So, I realize that many schools are trying to minimize lecture hours, but the truth is that this modality will not likely ever go away completely.  As such, I've made a draft of some guidelines for lecturers in the course I co-direct based on common mistakes I've seen.  Implicit in these guidelines is the idea that I have a lot of rotating lecturers, and some of these issues can be avoided by decreasing volume of presenters.  However, we're stuck in a cycle where I can't really easily change that in the next year.  Please share in the comments if you have other things I haven't addressed.

*Note - NSB stands for neuroscience and behavior -a 9 week introductory course on neuroanatomy, neurophysiology with and some clinical neurology, psychiatry, and neurosurgery.

---------------

NSB lecturer guide:

Please adhere to the following guidelines when giving lectures for NSB.  These are general trends which I have noticed over the last few years in terms of best practices and things to avoid if possible.  In general, please remember that these are students who have spent the last year and a half going to lots of lectures.  Things which seem trivial to you as you only do one or two lectures, for them are agonizing as they see these issues over and over again for two years.

1)            Do look over the slides of lecturers who are talking about related topics to your lecturer.

- Helps a lot with keeping continuity through the course.  It would be nice to not have you say, “I’m not sure if you have been over this before or not.”  If you are not sure, please look it up or ask.

2)            Don’t ever say, “This will not be on the test.” 

- Sometimes it is a minor point in your lecture, but it is a major point in a prior lecture (again why it helps to talk to others) or a later lecture (maybe even in the next block).  OK to say, “This is a minor point for the purposes of this lecture,” or “For this lecture, this is primarily FYI.”  We tell the students that everything mentioned in class is potentially testable.  If you don’t want us to potentially test it, don’t mention it at all.  If you must say this, talk to the course director to be sure it REALLY will not be on the test.

3)            Try to stay away from disclosure slide jokes.

- The first person who puts up the “I am actively looking for people to give me money so I can disclose it” slide is funny.  The next five are not so much.

4)            Please know how much time your lecture is scheduled for, and try to stick to that.

- Our general lecture block time is 50 minutes.  In general, we plan our lectures to end 10 minutes prior to the next lecture.  So if your lecture is at 10, you should plan to be done by 10:50.  If you bleed over, this makes everyone behind you have to modify their talks.  If you have a lot of slides left, and are running short on time, consider stopping where you are at, and recording the end of the lecture to be posted online.

- If you are the last lecturer of the day, it is OK to keep going (within reason).  However, please announce that it is OK for students who need to leave to be able to get up and go.  Some students have tight commute times to get to community preceptor sites by 1 PM or have to walk across campus to do OSCE testing over the noon hour.  Also keep in mind that there are noon talks which occur periodically in the lecture hall.

5)            If you are not skilled at PP or basic functions of the audio/visual equipment, please learn minimum functions.

- For PP, you should be able to start and stop your presentation, restart  from the middle of a presentation, go backwards and forwards using only keyboard, understand what the purpose of a right click is, start and stop video presentations, be able to disengage auto-advancing of slides.

- For A/V, you should be able to turn on and off overhead projector, mute/ unmute the screen, turn on/off the mic,  Turn mic up/ down using front panel controls.

- If you are uncomfortable with this, talk to TSO, and they can help you learn how to do these things.

6)            Do feel free to experiment with audience participation techniques.

- Using clickers for in-lecture quizzing, use of pause for students to work through a problem together, and other techniques are great ways to get students involved in the lecture.

What does your academic rank have to do with how much you teach?

I've had this conversation a few times in my time as a clerkship director/ course director in medical school.  I have a spot that comes open for some one to do a lecture or teach a session, or I want to start a new educational activity.  I ask around to see who is available either from my department or other departments.  Often the name I am given is added with the caveat, "...because they need to get some more education activities on their CV for promotion purposes."  I'm not saying this is entirely bad.  What I am saying is that this statement has some underlying tones that I want to say overtly.

Part of what drives this phenomenon is the more junior faculty truly may not have done as much in education as they probably should, and as more active-learning techniques are being employed, there is a need for more facilitators.  Thus, there is a sense that recruitment needs help.  This is true.  On the other hand, as this incentive only works through the promotion cycle, it doesn't work for full professors.  Thus, the downstream (perhaps unintentional) consequence of this, is that full professors tend to step aside so that the junior faculty can carry the load.  This leaves the educational load on the younger faculty.  I don't think this is all bad.  Many young faculty are outstanding educators.  My concern is that once the promotion cycle has moved through its course, there is not the incentive to contribute.  And there are potentially good educators who are not educating any more.

I understand that this does not apply to all senior faculty.  I also understand that if a senior faculty member is not doing as much in education, it may be that they have taken on more administrative tasks or their research endeavors have become a more central focus.  I'm OK with that also.  How time is spent in an academic position is based on a whole host of factors, and many of them are external to the faculty member themselves.

I'm just asking if we should start rethinking some of the unintended consequences of how we currently do promotion.  Maybe a three to four step process (instructor, assistant professor, associate professor, full professor) doesn't fit as well in the 21st century.  Maybe the milestones project won't end with the end of residency for academic physicians.  Just a thought.  I'd appreciate to hear what you think.