What is Critical Appraisal?
Once you have asked the clinical question and searched for evidence, Assessment 2 asks you to critically appraise the evidence you have found. This is a really important process because it allows you to:
- cut down on information overload,
- disregard studies which have scientific flaws, and
- find studies relevant to your situation.
In the Evidence-Based Practice process, and especially in the process of evaluating primary research, we need to go beyond the usual CRAP test to also critically appraise certain aspects of the evidence. Even when we have found a study addresses our keywords and concepts, certain questions need to be addressed, especially:
- Is the study relevant to our clinical question?
- How well (scientifically) was the study done, especially taking care to eliminate bias?
- What do the results mean and are they statistically valid (and not just due to chance)?
Introduction to Critical Appraisal
- Introduction to Critical Appraisal - A 4:11 min. YouTube video (actually it's more like a podcast) which looks at the background to critical appraisal, what it is, and why we do it.
- How to read a paper: the basics of Evidence-Based Medicine by Trish Greenhalgh - This book was originally a series of articles in the British Medical Journal. The link here is to the 4th edition (2010) as an eBook.
- Critically appraise the evidence - Part 2: EBP Step 3 in the eBook by Glasziou, Salisbury, & Del Mar quoted above. Includes an example of critical appraisal of a real RCT into the use of elastic stockings to prevent deep-vein thrombosis (DVT) on long aeroplane flights.
Critical Appraisal Tools
You have been asked in Assessment 2 to use a critical appraisal tool to critically appraise each of the research papers you have identified. Here are some commonly used tools:
- Critical Appraisal Skills Programme (CASP) is part of Better Value Healthcare based in Oxford, UK. It includes a series of checklists, suitable for different types of studies and designed to be used when reading research.
- The Equator Network is devoted to Enhancing the QUAlity and Transparency Of health Research. Among other functions, they include a Toolkit for Peer Reviewing Health Research which is very useful as a guide for critically appraising studies.
- Johanna Briggs Institute provides access to critical appraisal tools, a collection of checklists that you can use to help you appraise or evaluate research.
- Critical Appraisal Tools (CEBM) - This site from the Centre of Evidence Based Medicine includes tools and worksheets for the critical appraisal of different types of medical evidence.
- Critical Appraisal Tools (iCAHE) - This site from the International Centre of Allied Health Evidence (at the University of South Australia) has a range of tools for various types of studies.
When deciding whether to include a study in your results, it is worth checking some of these factors.
- Relevance - It's easy to do this by comparing the PICO used in the study to the PICO of your clinical question. Is it close enough to make the results relevant?
- Bias - "Bias is the degree to which the result is skewed away from the truth" . Bias can occur in many ways, and can occur in the selection of subjects for study, allocation to groups, and measurement of results, but the best way to prevent it is to keep details of interventions hidden from participants and also researchers (Glasziou, Salisbury, & Del Mar, 2007, p. 75). This is called blinding. How much is this study prone to bias?
- Confounding factors - "Confounding factors are patient features and other possible causal factors, apart from the one that is being measured, that can affect the outcome of the study." These can be eliminated or reduced by ensuring that groups are matched as closely as possible at the start of the study, and managed in the same way in the course of the study (Glasziou et al., 2007, p. 76).
- Trustworthiness - the RAMMbo mnemonic can help you to work out how trustworthy a study is:
|R||Recruitment||Were the subjects in the study representative of the target population? Were there enough subjects to make the study valid?|
|A||Allocation||Were the subjects randomly (and "blindly") allocated to groups?|
|M||Maintenance||Was the status of the study group and the control group maintained throughout the trial? Were they treated the same way apart from the intervention?|
|Mbo||Measurement (blinding, objective measures)||Were the subjects (and researchers) blinded to the intervention and were the outcomes measured objectively? Was bias eliminated as much as possible?|
Source: Glasziou, P., Del Mar, C., & Salisbury, J. (2009). Evidence-based practice workbook: Bridging the gap between health care research and practice (2nd ed.). Retrieved from ProQuest Ebook Central