Augmented reality for neurological education simulations

I am laying down a challenge for app developers out there who know more about programming than I do.  This challenge comes from a day-long IAMSE meeting course I attended over the weekend on state-of-the art medical simulation tools.  What I saw was some pretty cool simulation is available today to replicate many physical signs and to help train on various procedures.  These simulations have come a long way from when all Harvey could do was teach you how to pick up a murmur consistent with mitral stenosis.  Now you can check blood pressure, pupillary response, breath sounds, and the mannequin can even talk to you.

The trouble (from a neurologist perspective) is that current simulation is great for cardiopulmonary physiology and simulation, but it leaves a void for the neurological exam.  It can teach laproscopic surgery, mimic a prostate nodule on DRE, and a lot of other things.  But aside from pupils and having the machine shake to mimic a seizure (which I haven't seen, but from the description, it sounds like a very large Tickle-Me-Elmo type of convulsion - ie all trunk movement and not much arm or leg movements), the neurological exam is as yet uncovered.  I think a lot of that comes from the fact that the neurological exam will require pretty advanced robotic arms and legs to mimic things like fine finger movements, and strength testing.  Hence, essentially you can equilibrate an essentially comatose person's exam for the most part.

I see an opportunity for augmented reality to step in while the robotic simulation takes time to become more sophisticated and cheaper.  I could imagine using a real person as a simulated patient sitting in a chair, or a simulation mannequin in a gurney, and have the student hold a tablet up to the person so that the view screen is over the torso.  Then an augmented reality protocol could take the image of the arm from the simulation, and overlay a realistic-looking tremor.  Or you could overlay realistic ataxia with heel to shin testing.  Or you could overlay a realistic tongue deviation, tongue fasiculations, or palate deviation.  Thus, you could more efficiently create a high fidelity simulation with neurological deficits.  I've asked my bioengineer friend about this, and he said it could probably be done, it'd just take money to get off the ground.

So, there's my challenge.  Create an augmented reality neurology exam simulation.  I'd be interested to hear if anyone is already developing something like this, or if someone if any app makers would be interested in making this happen.

Use of Tablets in Medical Education - What we can learn from Manchester

I've been watching a series of presentations by students at Manchester Medical School where they discuss their use of the iPad in their studies.  This is the end result of a project where students where issued an iPad in a pilot project at the beginning of this year.  I don't think any of the presentations in and of themselves is ground-breaking.  What I think is very innovative about this program is to have the students take the technology and apply it to problems they identify.  This is not a top-down approach where an instructor is listing apps the students can use, and then evaluating whether the students do what they are told. 

This is really a problem-solving exercise.  It's learner-centered learning at it's very core.  Give a student a tool which has over 30,000 apps available plus web capability, now the students need to go and figure out how to best use it.  First, they all identified problems they had in the past - forgetting important papers at home, having notes highlighted beyond recognition, and inability to physically lug all those textbooks.  They sifted through the app landscape, and came up with some remarkable ways to use the technology.  This is crowd-sourcing at its best.  This is truly the future of technology in medical education.  It's not about the downloading the coolest toy out there and jamming it into a curriculum to make it do something, its about finding the right tool to use to solve the educational problem in front of you.  The students found ways to get around problems with creating and filing notes, filing reading to do later, communicating log data to their supervisor, and creating study aids for themselves and their classmates.  If all the students at Manchester did this project next year, think of the innovations they could produce.  Now think about all the students in the UK, or across nations.  Again, crowd-sourcing at its best.

As Prof Freemont explains in his introduction to the program, this program is not about one particular format.  They chose the iPad for reasons he outlines, but you could likely accomplish similar feats with an army of portable laptops, android tablets, iPads, or whatever the next new thing will be.  I do appreciate the spirit of their experiment.  And, I'd be happy to come personally see what's going on in Manchester.  Maybe sometime next year during football season.

Twitter Live Meeting Stream as a Self-reflection Tool

I doubt this post will come as a surprise to many social media gurus out there, but it is something I fully realized only last week.  I've posted to Twitter more regularly from meetings lately.  It is decidedly a skill which I'm still working on mastering.  I think it is a very powerful tool to use while in a meeting to connect with those in the room with you, and also to disseminate information with those not at the meeting.  However, I didn't really get that it could also be useful to me as a self-reflection tool.  I've seen Twitter used more intentionally as a self-reflection tool in an education setting as discussed in this very nice slide presentation posted by Dr. Noeline Wright.  I'd also seen twitter chats put together using Storify or similar sites.  I always thought these things were for the people who weren't in the room, and weren't posting live.

 Then I was sitting at the Pacific Northwest Basal Ganglia Coterie (Parkinson's doctors and scientists) meeting this last weekend next to a fellow conference goer.  He was getting preparing to jot down some notes, and looked at my laptop which was open to Hootesuite.  I had presented about my Twitter account to this group before, so he figured out pretty quickly what live Tweeting from a meeting would entail.  But then, he made the assumption that I was doing the Tweeting primarily for myself as a record that I could go back to look at later.  Again, maybe I'm just a dunderhead, but when I've live Tweeted meeting updates before, I usually didn't think it was for me.  I was thinking about those that may read my stream and learn from it.  I've seen data that reflection is better for retention of lecture material, and yet I didn't put that together.  I went back through my stream at the end of the conference, and hopefully more of the information will stick because of it.

Now that I've figured this out, and I plan to go back through my Twitter feed intermittently as a reflection tool.  The Twitter feeds from meetings may have other valuable information to mine including using it as a a way to prove that you were actively mentally participating in a CME event.  I could be used to evaluate the CME, and also if an presenter has a rich group of streams to look at, it can give them loads of information about the audience for future talk planning.  Who knew all this could come from live Tweeting at a meeting?

EBM evaluation tools applied to medical student assessment tools

I remember back to the days when I was a fresh medical student taking those first classes in biochem, anatomy, and cell biology.  I learned a ton, and honestly I draw on this knowledge-base daily when I'm taking care of patients.  I also remember that the assessments methods used during my first year of medical school were not the greatest (in the opinion of a person who was teaching high school physics and chemistry 3 months before entering med school).  The number of assessments used in med schools has risen over the last 15 years since I was an M1.  However, with a rise in number of choices, comes responsibility to utilize the right choice.  Another way to look at this from an pedagogical standpoint is are the assessments really measuring the outcomes you think they are measuring.  To attempt to help the medical educator with this dilemma, I came up with the idea that you can apply a well-known paradigm used to evaluate evidence-based medicine (EBM) to evaluate a student assessment.  The EBM evaluation methods I've been most familiar with is outlined by Straus and colleagues in their book, Evidence-Based Medicine: How to Practice and Teach EBM, copyright 2005.

Here's my proposed way to assess assessment:

1)  Is the assessment tool valid?  By this we need to be sure that our measurement tool is reliable and accurate in being able to measure what we want it to measure.  The standardized (high-stakes) examinations like MCAT, USMLE and board certification examinations are expensive not because these companies are rolling in cash, but because it takes people LOTS of time to validate a test.  Hence, most home-grown tools are not completely validated (although some have been).  To be validated an assessment has to be likely to give similar results if the same learner takes the test each time.  It also has to accurately categorize the level of proficiency of the learner at the task you are measuring.

For example, let's say I have an OSCE to assess whether a learner can counsel a young woman of child-bearing age on her options for migraine prophylaxitic medications.  For my OSCE to be valid, I need to look for reliability and accuracy.  Does the OSCE predictably identify learners who do not understand that valproate has teratogenic potential, and don't discuss this with a standardized patient?  You also want to know if it is accurate, in other words does your scoring method give similar results if multiple faculty who have been trained on how to use the tool score the same student interaction?  To truly answer these questions on an assessment, it takes multiple data points for both raters and learners - hence why it takes time and money, and also why most assessments are not truly validated.

The best way to validate is to measure the assessment against another 'gold standard' assessment.  How well does your assessment work compared with known validated scales.  Unfortunately, there aren't as many 'gold standard' assessments outside of the clinical knowledge domain in medical education (although it is getting better).

2)  Is the valid assessment tool important?  Here we need to talk about whether the difference seen in the assessment is actually a real difference.  How big is the gap between those who just passed without trouble, just barely passed, and those who failed to meet the expected mark?  Medical students are all very bright, and sometimes the difference between the very top and the middle is not that great a margin (even if it looks like it on the measures that we are using).  I think the place where we trip up here sometimes is in assuming that Likert scale numbers have a linear relationship.  Is a step-wise difference from 3 to 4 t o 5 on the scale set up on the clinical evaluations a reasonable assumption, and is the difference between a 4 and a 5 really important?  It might very well be that this is true, but it will be different for every scale that we set up.  I've never been a big fan of using Likert rating scores to directly come up with a percentage point score unless you can prove to me through your distribution numbers that it is working.

3)  Is this valid, important tool able to be applied to my learners?  I think this step involves several steps.  First, are you actually measuring what you'd like to measure?  A valid, reliable tool for measuring knowledge (typical MCQ test) unless it is very artfully crafted will not likely assess clinical reasoning skills or problem-solving.  So, if your objective is to teach the learner how to identify 'red flags' in a headache patient history, is that validated MCQ the best assessment tool to use?  Is it OK that that learner can pick out 'red flags' from a list of distractors, or is it a different skill set to be able to identify this in a clinical setting?  I'm not saying MCQ's can never be used in this situation, you just have to think about it first.

Second, if you are utilizng a tool from another source and you did not design it for your particular curriculum, is the tool useful for the unique objectives?  Most of the time this is OK, and cross-fertilization of educational tools is necessary due to the time and effort bit.  But, you have to think about what you are actually doing.  In our example of the headache OSCE, let's say you found a colleague at another institution who has an OSCE set up to assess communication of differential diagnosis and evaluation to a person with migraine who is worried they have a brain tumor.  You then apply that to your clerkship, but you are more interested in the above scenario about choice of therapy.  Will the tool still work when you tweak it?  It may or may not, and you just need to be careful.

Hopefully you've survived to read through to the end of this post.  Hopefully you learned something about assessment in medical education, and you found the EBM-esque approach to assessment evaluation useful.  My concern is that in general, not enough time is spent considering these questions, and more time is spent on developing the content then on assessment.  I'm guilty of this as well, but I'm trying to get better.  Thanks for reading, and feel free to post comments/thoughts below.