LibGuides: Systematic Reviews/Evidence Synthesis: Quality Assessment/Risk of Bias Tools

Studies included in a systematic review need to be evaluated for study quality and risk of bias. Per Page et al., (2021) "Risk of bias refers to the potential for study findings to systematically deviate from the truth due to methodological flaws in the design, conduct or analysis." Risk of Bias tools are not measuring bias, but rather the risk of bias inherent in a given study.

What Quality Assessment Tool Should I Use? A Practical Guide for Systematic Reviews Authors See Appendix 1 for a handy chart of appropriate reporting guidelines for each study type/design. (Delavari et al., 2023).

Repository of Quality Assessment and Risk of Bias Tools
A handy resource from Duke University Medical Center Library for finding and selecting a risk of bias or quality assessment tool for evidence synthesis projects. Download spreadsheet for full functionality.

Hint: Sort the spreadsheet by study type to find the appropriate risk of bias tool for your study:

Instructions for Use

This project was created and is maintained by Duke University's Medical Center Library & Archives. Quality Assessment and Risk of Bias Tool Repository © 2021 by Leila Ledbetter and Stephanie Hendren is licensed under Attribution-NonCommercial 4.0 International.To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/

Some common Quality Assessment/Risk of Bias Tools:

CASP Checklists
Critical Appraisal Checklists from the Critical Skills Appraisal Programme, UK.
Downs and Black
See Appendix. Provides both an overall score for study quality and a numeric score out of a possible 30 points. It has five sections. Administration of the tool can happen either within a systematic review process, or as a quality assessment tool for individual articles. (Downs & Black, 1998).

JBI Critical Appraisal Tools Joanna Briggs Institute (JBI) is an independent, international, not-for-profit researching and development organization based at the University of Adelaide, South Australia. Includes a number of critical appraisal tools including Checklist for Randomized Controlled Trials.

Modified Downs and Black
See Appendix 4. (MacLehose et al., 2000).
more...less...
In their cited paper, they state they made modifications to quantify "the relationship between study quality and effect size.Modifications were made, or additional questions developed, to quantify the extent to which factors likely to influence effect size (e.g. adjustment for confounders, resolution with which confounding variables were measured) were addressed by a study. The original questions about external validity were also merged into a single question. By assessing all the studies on the same question about external validity, we sought to obtain some indication of the external validity of different study designs and hence the extent to which differences in effect size could be attributed to this factor."
Newcastle Ottawa Scale
Developed to assess the quality of nonrandomized studies including case control or cohort studies. (Wells et al., 2000).
RoB 2.0
A revised Cochrane risk of bias tool for randomized trials. Suitable for individually-randomized, parallel-group, and cluster- randomized trials
ROB-ME
A tool for assessing Risk Of Bias due to Missing Evidence in a synthesis.
Matthew Page, Monash University. Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

More critical appraisal tools:

Best Bests Critical Appraisal Worksheets
Critical appraisal checklists for various study types.
Critical Appraisal Tools
Critical appraisal questions to ask and worksheets from the Centre for Evidence-Based Medicine at Oxford University.
University of Glasgow EBP Checklists

robvis
A web app designed to for visualizing risk-of-bias assessments performed as part of a systematic review. The tool creates “traffic light” plots of the domain-level judgements for each individual result; and weighted bar plots of the distribution of risk-of-bias judgements within each bias domain.