Use of Tablets in Medical Education - What we can learn from Manchester

I've been watching a series of presentations by students at Manchester Medical School where they discuss their use of the iPad in their studies.  This is the end result of a project where students where issued an iPad in a pilot project at the beginning of this year.  I don't think any of the presentations in and of themselves is ground-breaking.  What I think is very innovative about this program is to have the students take the technology and apply it to problems they identify.  This is not a top-down approach where an instructor is listing apps the students can use, and then evaluating whether the students do what they are told. 

This is really a problem-solving exercise.  It's learner-centered learning at it's very core.  Give a student a tool which has over 30,000 apps available plus web capability, now the students need to go and figure out how to best use it.  First, they all identified problems they had in the past - forgetting important papers at home, having notes highlighted beyond recognition, and inability to physically lug all those textbooks.  They sifted through the app landscape, and came up with some remarkable ways to use the technology.  This is crowd-sourcing at its best.  This is truly the future of technology in medical education.  It's not about the downloading the coolest toy out there and jamming it into a curriculum to make it do something, its about finding the right tool to use to solve the educational problem in front of you.  The students found ways to get around problems with creating and filing notes, filing reading to do later, communicating log data to their supervisor, and creating study aids for themselves and their classmates.  If all the students at Manchester did this project next year, think of the innovations they could produce.  Now think about all the students in the UK, or across nations.  Again, crowd-sourcing at its best.

As Prof Freemont explains in his introduction to the program, this program is not about one particular format.  They chose the iPad for reasons he outlines, but you could likely accomplish similar feats with an army of portable laptops, android tablets, iPads, or whatever the next new thing will be.  I do appreciate the spirit of their experiment.  And, I'd be happy to come personally see what's going on in Manchester.  Maybe sometime next year during football season.

Twitter Live Meeting Stream as a Self-reflection Tool

I doubt this post will come as a surprise to many social media gurus out there, but it is something I fully realized only last week.  I've posted to Twitter more regularly from meetings lately.  It is decidedly a skill which I'm still working on mastering.  I think it is a very powerful tool to use while in a meeting to connect with those in the room with you, and also to disseminate information with those not at the meeting.  However, I didn't really get that it could also be useful to me as a self-reflection tool.  I've seen Twitter used more intentionally as a self-reflection tool in an education setting as discussed in this very nice slide presentation posted by Dr. Noeline Wright.  I'd also seen twitter chats put together using Storify or similar sites.  I always thought these things were for the people who weren't in the room, and weren't posting live.

 Then I was sitting at the Pacific Northwest Basal Ganglia Coterie (Parkinson's doctors and scientists) meeting this last weekend next to a fellow conference goer.  He was getting preparing to jot down some notes, and looked at my laptop which was open to Hootesuite.  I had presented about my Twitter account to this group before, so he figured out pretty quickly what live Tweeting from a meeting would entail.  But then, he made the assumption that I was doing the Tweeting primarily for myself as a record that I could go back to look at later.  Again, maybe I'm just a dunderhead, but when I've live Tweeted meeting updates before, I usually didn't think it was for me.  I was thinking about those that may read my stream and learn from it.  I've seen data that reflection is better for retention of lecture material, and yet I didn't put that together.  I went back through my stream at the end of the conference, and hopefully more of the information will stick because of it.

Now that I've figured this out, and I plan to go back through my Twitter feed intermittently as a reflection tool.  The Twitter feeds from meetings may have other valuable information to mine including using it as a a way to prove that you were actively mentally participating in a CME event.  I could be used to evaluate the CME, and also if an presenter has a rich group of streams to look at, it can give them loads of information about the audience for future talk planning.  Who knew all this could come from live Tweeting at a meeting?

EBM evaluation tools applied to medical student assessment tools

I remember back to the days when I was a fresh medical student taking those first classes in biochem, anatomy, and cell biology.  I learned a ton, and honestly I draw on this knowledge-base daily when I'm taking care of patients.  I also remember that the assessments methods used during my first year of medical school were not the greatest (in the opinion of a person who was teaching high school physics and chemistry 3 months before entering med school).  The number of assessments used in med schools has risen over the last 15 years since I was an M1.  However, with a rise in number of choices, comes responsibility to utilize the right choice.  Another way to look at this from an pedagogical standpoint is are the assessments really measuring the outcomes you think they are measuring.  To attempt to help the medical educator with this dilemma, I came up with the idea that you can apply a well-known paradigm used to evaluate evidence-based medicine (EBM) to evaluate a student assessment.  The EBM evaluation methods I've been most familiar with is outlined by Straus and colleagues in their book, Evidence-Based Medicine: How to Practice and Teach EBM, copyright 2005.

Here's my proposed way to assess assessment:

1)  Is the assessment tool valid?  By this we need to be sure that our measurement tool is reliable and accurate in being able to measure what we want it to measure.  The standardized (high-stakes) examinations like MCAT, USMLE and board certification examinations are expensive not because these companies are rolling in cash, but because it takes people LOTS of time to validate a test.  Hence, most home-grown tools are not completely validated (although some have been).  To be validated an assessment has to be likely to give similar results if the same learner takes the test each time.  It also has to accurately categorize the level of proficiency of the learner at the task you are measuring.

For example, let's say I have an OSCE to assess whether a learner can counsel a young woman of child-bearing age on her options for migraine prophylaxitic medications.  For my OSCE to be valid, I need to look for reliability and accuracy.  Does the OSCE predictably identify learners who do not understand that valproate has teratogenic potential, and don't discuss this with a standardized patient?  You also want to know if it is accurate, in other words does your scoring method give similar results if multiple faculty who have been trained on how to use the tool score the same student interaction?  To truly answer these questions on an assessment, it takes multiple data points for both raters and learners - hence why it takes time and money, and also why most assessments are not truly validated.

The best way to validate is to measure the assessment against another 'gold standard' assessment.  How well does your assessment work compared with known validated scales.  Unfortunately, there aren't as many 'gold standard' assessments outside of the clinical knowledge domain in medical education (although it is getting better).

2)  Is the valid assessment tool important?  Here we need to talk about whether the difference seen in the assessment is actually a real difference.  How big is the gap between those who just passed without trouble, just barely passed, and those who failed to meet the expected mark?  Medical students are all very bright, and sometimes the difference between the very top and the middle is not that great a margin (even if it looks like it on the measures that we are using).  I think the place where we trip up here sometimes is in assuming that Likert scale numbers have a linear relationship.  Is a step-wise difference from 3 to 4 t o 5 on the scale set up on the clinical evaluations a reasonable assumption, and is the difference between a 4 and a 5 really important?  It might very well be that this is true, but it will be different for every scale that we set up.  I've never been a big fan of using Likert rating scores to directly come up with a percentage point score unless you can prove to me through your distribution numbers that it is working.

3)  Is this valid, important tool able to be applied to my learners?  I think this step involves several steps.  First, are you actually measuring what you'd like to measure?  A valid, reliable tool for measuring knowledge (typical MCQ test) unless it is very artfully crafted will not likely assess clinical reasoning skills or problem-solving.  So, if your objective is to teach the learner how to identify 'red flags' in a headache patient history, is that validated MCQ the best assessment tool to use?  Is it OK that that learner can pick out 'red flags' from a list of distractors, or is it a different skill set to be able to identify this in a clinical setting?  I'm not saying MCQ's can never be used in this situation, you just have to think about it first.

Second, if you are utilizng a tool from another source and you did not design it for your particular curriculum, is the tool useful for the unique objectives?  Most of the time this is OK, and cross-fertilization of educational tools is necessary due to the time and effort bit.  But, you have to think about what you are actually doing.  In our example of the headache OSCE, let's say you found a colleague at another institution who has an OSCE set up to assess communication of differential diagnosis and evaluation to a person with migraine who is worried they have a brain tumor.  You then apply that to your clerkship, but you are more interested in the above scenario about choice of therapy.  Will the tool still work when you tweak it?  It may or may not, and you just need to be careful.

Hopefully you've survived to read through to the end of this post.  Hopefully you learned something about assessment in medical education, and you found the EBM-esque approach to assessment evaluation useful.  My concern is that in general, not enough time is spent considering these questions, and more time is spent on developing the content then on assessment.  I'm guilty of this as well, but I'm trying to get better.  Thanks for reading, and feel free to post comments/thoughts below.

Senioritis in medical school - How to motivate the abulic state

It's that time of year where med students throughout the US shake hands with the Dean, and pull their tassels in unison from the right to the left.  In the months leading up to the tears and endless photo ops that mark any graduation, the students are finishing up the last few rotations of their medical school career.  Although some students retain focus on their swan-song rotations, in every heart there is always the lure of looking beyond the present rotation to the allure of residency in all its glory.  Some have more trouble than others with maintaining drive at the end of the final year.  Most students post-match are taking electives or required courses which are not directly aligned with their chosen field of study.  This makes some sense for getting a Dean's letter together and positioning one's self to be a desirable residency applicant.  Post-match all of this does not seem to matter as much.  Indeed, there has been discussion among academic educators that there is a missed opportunity in the fourth year of medical school largely based on articles like this one from Dr. Lyss-Lerman and colleagues which outlined residency directors view of how well the fourth year is working to prepare for residency.

So, I have a bunch of fourth year students in my neurology clerkship, in fact I have only fourth-year students with the exception of a few third-years that can take neurology as an elective in one block in November.  These students are largely not going into neurology.  How do I try to keep them engaged in our neurology rotation.  (Full disclosure - I'm fully aware I can learn more about how to do this.  I definitely still get some students for whom my little tricks don't work.  This is partly why I'm starting this discussion so that we can all learn from each other).  Here are some of my ideas:

- Have them create their own goals - In orientation, I encourage the students to come up with their own course goals and objectives.  I have prepared goals and objectives, and they are held accountable to those, but there may be specific areas they want to focus on as an area of weakness or as an area which is important for their specialty.  I tell them that in residency, you will not always have clear objectives which are overtly given to you for every rotation.  Thus, I made a habit in residency of picking 2-3 key things I wanted to learn.  When I was on cardiology as an intern, I wanted to sharpen my EKG reading skills and my cardiac exam skills.  Thus, I had something to focus on while taking care of those patients.  Intrinsically created goals are more motivating.  I encourage the students to follow that model as they move on in their career.

-  Encourage exploration of topics related to their field - This is partly a student-led issue, and partly faculty development.  Often students will stay engaged if the faculty recognizes what they are going into, and discusses aspects of a neurological case which is of interest to the student.  For example, we had an OMFS fellow rotating through the neurology clerkship, and I took him aside to discuss a case of a siallorhea I was seeing in the setting of neurodegenerative disease.  Sure it's important for him to know how to treat those diseases from a neurologic standpoint, but he's going to be more interested in the salivary issues.  This can then be used as a doorway to get them to be interested in the rest of the disease.

- Try using games - I haven't used this in my clerkship yet, but as a medical student and a resident, we had an attending (Dr. Harold Adams) who would play Neurojeapordy several times during the rotation.  Students were put into teams, and asked neuro-trivia questions about neuroanatomy, neurological differential diagnosis/treatment, and neurological history.  As a student (and a resident) I really enjoyed this.  It's a way to get students to want to read up on disease states, etc.

- Scare the bejeezers out of them - I will often also play the card that in only two to three short months, they will be responsible for caring for patients on their own (in a supervised fashion initially).  Their signature will mean something, and when someone in their care has a neurological problem, they will likely be the first person to evaluate the situation.  Starting on July 1.  Most students understand this logic.

These are just a few ideas I've used.  Any other thoughts on how to motivate the post-match senior on a required rotation?  Leave them in the comments below!

AAN annual meeting blog promotion ideas accepted

From AAN Annual Meeting website referenced in text

If you are not a neurologist, you probably weren't aware that the American Academy of Neurology Annual Meeting is this next week in New Orleans, LA.  The meeting brings to together neurologists from around the globe to learn from and connect with colleagues.  A major portion of the meeting for myself is devoted to attending medical education sessions and meetings with other neurology educators.  There will be a full day meeting of clerkships and residency directors as well as a half-day session on education topics and a poster and platform presentations.

As I go to the meeting, I'd like to be able to not only learn some things from the meeting, but also do some shameless self-promotion for my blog.  As my blog is fairly newly established, I don't really feel like it is at a stage where I could have made a poster or abstract about its relative worth to the community of educators.  As such, I may have missed an obvious outlet for creating interest and awareness in my blog.  I was wondering if others who have blogs could comment on creative ways to let people at meetings like this know that your blog exists while being relatively subtle.  I'm thinking the strategy of going to the open mic and asking a question about an unrelated presentation that ends with the statement, "...I'm very interested in this as I'd like to include it in my medical education blog found on neuronerd.com."  Are there ways of spreading blog love at meetings?  Thanks for the advice.

Virtual hospitals - The future of medical simulation?

I was flipping through Facebook the other day, and saw a video posted by a friend.  It was over 17 minutes long which is eternal in the world of Facebook videos, but I thought I'd give it a try as it looked interesting.  It ended showing people from 'The Gadget Show' making a simulator which not too long ago would have been pure fantasy.  They built a tent with 360 video output capability that also has a 360 degree treadmill to allow you to move in the virtual world by walking as you would in real life.  They also hooked up an X-Box Kinnect sensor to pick up other body movements.  They through in a few other cool add-ons, and they ended up with a truly immersive environment for a first person shooter game.  You can watch the video here.

I got to thinking that this technology is now available, and could be used in medical school to train physicians.  It's not yet at the level of being a holodeck, but it is closer than we've ever been before.  I could envision a program where there is an ambulatory office building, and the student has their own clinic to run where simulated patients come in be interviewed.  The physical examination is done through use of gestures mimicking what the real PE would be, or it could be coupled with a simulation manikin to elicit the physical findings.  Then the student has to go back to a virtual staffing room, and dictate the encounter, and order testing.  They then move on to the next patient.  If you had enough of these built (assuming in the future this type of technology gets cheaper), you could envision having a 'continuity clinic' set up completely in a simulator.  This might include seeing some of your regular patients back as they come through the emergency room for acute conditions or even going to the OR.  It could be as complex as there is time and money to create the scenarios.

I often thought in residency that it would be interesting to have an immersive simulated hospital where you could spend at least some of your time as a medical student or as a junior resident.  There you could have freedom to make some truly independent decisions and see what happens.  I think the advantages to something like this are obvious and are akin to the flight simulators that pilots use to train.  It will never replace time spent on the wards with skilled clinicians giving supervision and feedback.  I don't think the technology is there for a completely realistic medical simulation.  But it is getting closer. 

Life - Do med students have one? Work-life balance across generations

I gave a journal club last week discussing some general ideas about generational differences between the three main groups trying to work together in medical education: Boomers, Gen X, and Gen Y (or whatever it is your preferred term for this generation is).  As I was looking into the topic to prepare for this talk, one of the themes that kept popping up was the work-life balance theme.  In general, the common wisdom is that the Boomers value hard work, and are willing to sacrifice family life for career advancement.  Gen X and Gen Y tend to have less of a focus on work as a source of primary identity, and see much more value in maintaining balance between career and home.  The purpose of this blog post is not to decide whether this is indeed true or not.

What I'd like to spend a moment discussing is how this generational difference is creating conflict in the halls of medical schools.  Medical students are primarily Gen Y (although there are some Gen X in the mix).  Faculty who now populate Dean's office level positions are primarily Boomers, and course/clerkship directors are now Boomers with some Gen X filling the junior ranks.  So, what happens is that the Boomers remember their medical school life which was ruled by the Greatest Generation (even more value on work due to their experiences in the Great Depression).  The biggest place I've seen this conflict play out is in requests for time off or for changing a date to take a test.  The Boomers were given very little room to change their schedule.  I've talked to many of them, and the stories were essentially that if you wanted to take a day off during the clinical years for anything other than being near-death, there would be severe consequences (like repeating the entire clerkship).  Things were a little better for me, but not a lot.  I remember having friends in medical school who had a lot of trouble getting time off to attend weddings or family reunions.  There was minor grumbling, but we all decided it was a transient time, and this was preparing us somehow for the trials of residency.  And we kept telling ourselves that things would eventually get better.  We also had the usual weeks of vacation around the Holidays and Spring Break for some time off.  Everyone also had some lighter rotations, and the fourth-year comes with a much more flexible schedule.

Then along comes Gen Y.  They are much more vocal about their need for time off, and much more vocal about providing feedback on things that they are not in agreement with.  And they are now complaining primarily to the Boomers, who primarily don't want to hear about it.  I'm not so sure.  Maybe it's my Gen X roots showing or maybe I'm still close enough to being a student that I remember the bind it puts you in if your schedule is completely inflexible.  So, I'm wondering if maybe school policies for personal days off should be revisited.  I'm thinking most of the policies were set in place for a very different world, and haven't been changed much for 20-30 years.  With the advent of technology, it is possible to make up some assignments which may not have been possible to make up in the past.  There's also a different cultural norm emerging (or maybe I just think this should happen), and missing a wedding because you are assigned to spend a day in clinic is not an acceptable trade-off.

As a disclaimer - I've been told by several fourth-year students that as a clerkship director, I run a 'tight ship'.  To my mind I'm just doing what the school time off policy is telling me to do.  Our school policy is that student have 2 days off per year which can be used for attending a professional meeting or if they are ill/ have a family emergency.  All other time off is at the discretion of the clerkship director, and must be made up.  I'm not sure I have a perfect answer as to how to change the current policy, but maybe working with appropriate representatives from Gen Y, Gen X, and Boomers we can work together to figure something out.