Pearls and Pitfalls in Medical Education Research

Michael Gottlieb, MD, FACEP
Vice Chair of Research, Ultrasound Division Director
Department of Emergency Medicine
Rush University Medical Center, Chicago, IL
Twitter: @MGottliebMD, @RushEmergency, and @RushEMSono

Petra Duran-Gehring, MD, FACEP
Ultrasound Division Director
Department of Emergency Medicine
University of Florida - Jacksonville, Jacksonville, FL
Twitter: @UFJaxEM and @BoldCityUS

Medical education research is a common form of research performed by ultrasound fellows and faculty. This is not surprising, as many of us are involved with teaching learners of various levels on a regular basis. While many of the same skills and strategies from clinical research can be applied to medical education research, there are several distinct nuances to education research that can help ensure that your study is more impactful and publishable. In this article, we will highlight several key pearls and pitfalls to education-specific research.

1. Expand Your Literature Search

When conducting the literature search, it is advisable to look outside the medical education field for relevant literature that could inform your design or findings. This may include other fields, such as nursing, psychology, primary education, or even business. While commonly used for many searches, PubMed tends to primarily focus on medicine. Consider expanding your search using other search databases, such as CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Education Resources Information Center), or PsycINFO. GoogleScholar can also be helpful to conduct unrestricted searches. A reference librarian can also be useful, if you have access to one at your institution.

2. Use an Educational Framework

Many education journals will expect or require that the study be informed by an existing education theory or framework. Conceptual frameworks act as a lens by which we can improve our knowledge on learning and education. There are an abundance of theories or frameworks available from which to choose and you will likely find several that may be applicable to a given study. The theories are often first introduced in the introduction or methods and then applied to the study findings in the discussion. Some examples of relevant educational theories in ultrasound education could include cognitive load, Kolb experiential learning, deliberate practice, or somatosensory cueing. Those interested in learning more about educational theories may find the Education Theory Made Practical Series by Academic Life in Emergency Medicine (Available here: https://www.aliem.com/education-theory-made-practical-series/).

3. Perform a Needs Assessment

Prior to initiating the proposed intervention, perform a needs assessment of your learners. This often involves both a general needs assessment and a targeted (learner group-specific) needs assessment, though other models have also been described.^1-3 Make sure to consider all of the relevant end-users (including faculty, facilitators, and staff), not just learners, when you are designing the program to ensure you consider all relevant program needs.

4. Consider a Wider Range of Outcomes

Medical education studies often include primarily low level and narrow outcomes in their assessments (e.g., course preference, perceived value). Consider incorporating higher level outcomes, using frameworks such as Kirkpatrick to identify the range of learner impacts.⁴ The Kirkpatrick framework includes four levels. Level one is learner reaction (e.g., perception or preference for an ultrasound course). Level two focuses on learning (e.g., knowledge, skills, and attitudes resulting from an ultrasound course). Level three seeks to assess behavioral change (e.g., increased ultrasound use resulting from a course). Level four focuses on broader results or impact (e.g., improved patient care or throughput resulting from an educational intervention). While Kirkpatrick and other similar models focus primarily on the learner, consider expanding your outcomes to include a wider range of components (e.g., time, effort, and cost required to run the course). Established program evaluation frameworks, such as the realist, logic, or the context/input/process/product models may help guide this.⁵

5. Determine Your Study Design

Much like clinical research, educational research has specific categories or types of research publications and it is important to ponder the aim of your study when choosing the design. Curricular or educational innovations are similar to case reports in clinical research, and seek to share a novel educational model or curricula that would be useful to the educational community as a whole. Qualitative studies seek to understand a specific phenomenon by observing patterns or themes, are usually performed on smaller study populations, and present primarily non-numerical data, utilizing interviews, focus groups and open-ended questions. In contrast, quantitative studies use deductive reasoning, present numerical data, and look at larger populations.⁶ Mixed methods utilize a combination of more than one method type to assess a given outcome (e.g., a focus group and a survey). 

Conclusion

Medical education research aims to investigate teaching, learning, and assessment methods used in medical training, which can include curriculum development, assessment methods, teaching techniques and faculty development. Not only can this research inform your local medical school curriculum, teaching practices, and assessment techniques, but it can also improve education efforts more broadly. Therefore, every interaction with learners, whether at bedside scan rounds, medical student lectures or faculty development sessions has the potential to inspire a research question. So, the next time you are faced with a challenging educational experience, consider turning it into a medical education research question.

References

1. Kern DE. Overview: a six-step approach to curriculum development. Curriculum Development for Medical Education: A Six-Step Approach, Third Edition; 2015.
2. Chan TM, Jordan J, Clarke SO, Lawson L, Coates WC, Yarris LM, Santen SA, Gottlieb M; Emergency Medicine Health Professions Education Research Consortium (EMPERC). Beyond the CLAIM: A comprehensive needs assessment strategy for creating an Advanced Medical Education Research Training Program (ARMED-MedEd). AEM Educ Train. 2022;6(1):e10720. doi: 10.1002/aet2.10720.
3. Gottlieb M, Wagner E, Wagner A, Chan T. Applying Design Thinking Principles to Curricular Development in Medical Education. AEM Educ Train. 2017;1(1):21-26. doi: 10.1002/aet2.10003.
4. Kirkpatrick D. Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler; 1998.
5. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach. 2012;34(5):e288-99. doi: 10.3109/0142159X.2012.668637. PMID: 22515309.
6. Yarris LM, Deiorio NM. Education research: a primer for educators in emergency medicine. Acad Emerg Med. 2011;18 Suppl 2:S27-35. doi: 10.1111/j.1553-2712.2011.01189.x. PMID: 21999555.