Simulation Research

Picture of Kathleen Atchison
Re: Measuring Outcomes
by Kathleen Atchison - Wednesday, December 22, 2010, 10:53 PM

I don't have form I can send you.  We make a list of behaviors we want to observe, like handwashing or checking the patient's armband, and check whether or not the students do it during simulation.  Each checklist of behaviors in which you want to see improvement will be unique depending on your program/class objectives and desired learning outcomes.  Our's tend to focus on safety measures like the one's I mentioned above.

The hardest part is figuring out which behaviors will lead to improved patient care so that you can say that you time is well spent.  Here is a suggested process.

1. Identify what the problem is.  (staff unaware of new process, procedure not carried out correctly, complication rates needing reduction, etc.)

2.  Look for reasons why the problem exists (This usually requires teamwork and may uncover simple solutions.  This is where you find out which problems can/should be addressed using simulation.  This will also help you to think of realistic scenarios use to address the problem.)

3.  Identify what measurable behaviors will lead to problem reduction/resolution.  (This is the hard part and should be backed up by evidence in the literature unless there is no research on the subject - then you should conduct a research study. (-: )

4.  If information is identified as something that will reduce the problem, provide the information. (i.e. online learning, e mail, staff meeting, memo)

5.  If practice is identified as something that will reduce the problem, identify the level of practice. (i.e. knowledge based exercises, skills session, simulation - best for high risk or low volume situations, or real life) and plan activities. 

     a.  If simulation is identified as the best practice medium, design realistic scenarios to elicit, encourage, or reinforce the desired behaviors.

     b.   Design a simple checklist for documenting the occurance of key behaviors identified in #3 during simulation.  This can be a yes/no checklist or, if there are degrees of performance, a Likert scale.  Key behaviors documented can include wanted and unwanted behaviors.  (To determine the type of checklist you need, you should think about if you just want the behaviors to occur, if you want them to occur in a certain way, or if you want them to occur on certain occassions/at certain times.)

     c.  Test the scenarios on a small group of people to determine if they ilicit the desired bahavior while providing realism and opportunities for critical thinking.

     d.  Edit and retest scenarios as needed, and implement the program.

     e.  Evaluate outcomes at predetermined intervals and edit the plan as needed.

You will discover things you didn't know during the process and may change course at any point.  You may decide that the behavior you were looking at isn't the key behavior to achieve the desired outcome.  Or, you may realize that your method of education is not the best option.  (Sometimes a simple poster works wonders.) 

I hope this helps you to determine the outcomes/measurable behaviors you want and to develop a checklist that is effective in identifying if those outcomes are being met.