Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context (Burls, 2009). Critical appraisal of studies involves checking the quality, reliability and relevance of the studies you've selected to help answer your review question. Depending on the type of study you are evaluating you may use different evaluation tools. When evaluating studies, some questions to consider are:
- Has the study's aim been clearly stated?
- Does the sample accurately reflect the population?
- Has the sampling method and size been described and justified?
- Have exclusions been stated?
- Is the control group easily identified?
- Is the loss to follow-up detailed?
- Can the results be replicated?
- Are there confounding factors?
- Are the conclusions logical?
- Can the results be extrapolated to other populations?
-Questions adapted from University of Illinois, Chicago Library, Evidence-Based Medicine. - Evaluating a Study. CC BY-NC [Attribution-NonCommercial]
Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice, Second Edition by Call Number: Main-Swirbul ; RA427 .U84 2008ISBN: 9780071590389Publication Date: 2008-06-11Praised for its clear explanations of detailed statistical and mathematical principles, this text concisely covers both basic and advanced concepts in using evidence-based medicine, everything you need to deliver optimal patient care.
See chapters: Finding the Evidence and Why Study Results Mislead.
Appraising Quantitative vs Qualitative Research
The articles below provide a step-by-step appraisal on how to critique quantitative and qualitative research articles:
- Ryan, F., Coughlan, M. & Cronin, P. (2007). Step-by-step guide to critiquing research. Part 1: quantitative research. British Journal of Nursing, 16(11), 658-663.
- Ryan, F., Coughlan, M. & Cronin, P. (2007). Step-by-step guide to critiquing research. Part 2: qualitative research. British Journal of Nursing, 16(2), 738-744.
More on Critical Appraisal
- Tod, D., Booth, A., & Smith, B. (2021). Critical appraisal. International Review of Sport and Exercise Psychology, 1-21. https://doi.org/10.1080/1750984X.2021.1952471 This article helps to define critical appraisal, identify its benefits, discuss conceptual issues influencing the adequacy of a critical appraisal, and detail procedures to help reviewers undertake critical appraisals.
- Buccheri, R. K., & Sharifi, C. (2017). Critical Appraisal Tools and Reporting Guidelines for Evidence‐Based Practice. Worldviews on Evidence-Based Nursing, 14(6), 463–472. https://doi.org/10.1111/wvn.12258 The primary purpose of this paper is to help nurses understand the difference between critical appraisal tools and reporting guidelines.
Selected Critical Appraisal Tools
- AMSTAR 2 - A Critical Appraisal Tool for Systematic Reviews(Shea et al., 2017).
- Best Bests Critical Appraisal WorksheetsCritical appraisal checklists for various study types.
- CASP ChecklistsCritical Assessmentl Skills Programme (CASP) has appraisal checklists designed for use with Randomized Controlled Trials and other study types.
- Critical Appraisal ToolsCritical appraisal questions to ask and worksheets from the Centre for Evidence-Based Medicine at Oxford University.
- Downs & Black- Checklist for Measuring Study QualitySee Appendix (Downs & Black, 1998).
- Joanna Briggs Institute (JBI) Critical Appraisal ToolsJBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers.
- Johns Hopkins Evidence-Based Practice ModelThe Johns Hopkins Evidence-Based Practice model for Nurses and Healthcare Professionals is a powerful problem-solving approach to clinical decision-making and is accompanied by user-friendly tools to guide individuals or groups through the EBP process. Must fill out online form to request permission to download tools.
- Mixed Methods Appraisal Tool (MMAT)The MMAT is intended to be used as a checklist for concomitantly appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods studies) (Hong et al., 2018).
- GRADE
For appraising reviews of effectiveness.
- GRADE CerQUAL
For appraising reviews including qualitative research evidence.
- Repository of Quality Assessment and Risk of Bias ToolsA handy resource from Duke University' Medical Center Library for finding and selecting a risk of bias or quality assessment tool for evidence synthesis projects. Download spreadsheet for full functionality.