Systematic Reviews

A systematic review is a ‘study about studies’ which focuses on a specific research question that it must try to answer by using explicit, transparent and reproducible methods to identify, critically appraise and synthesize results of similar studies to generate evidence-based findings. (From Stephanie Roth’s Introduction to Systematic Reviews Course. June 2020).  

Identify: Review teams develop search strategies for multiple research databases and resources, and screen their findings against pre-specified eligibility criteria. 

Appraise: Once all the evidence has been located and screened, review teams evaluate the quality of the evidence, assessing to understand where more research may be needed. 

Synthesize: Finally, the review team brings together the different pieces of evidence, identifying trends, issues, or patterns in the data. The rigorous systematic review methodology makes these reviews a great tool for clinicians, educators, and stakeholders involved in decision-making or best practice recommendations. 

• Time-intensive: A typical systematic review can take 12-18 months to complete. 
• Collaboration-intensive: The systematic review’s methodology requires multiple researchers to screen and evaluate evidence. A team of at least three is required. 
• Protocol-based: Before beginning a systematic review, the team will pre-define their research question, expected outcomes, search strategy, and inclusion/exclusion criteria in a protocol. This helps mitigate potential bias or data manipulation.  
• Research-question based: Systematic reviews are rooted in clear, focused questions. Many reviews will use the PICO framework or something similar to build their question. 
• Exhaustive: The goal of a systematic review is to locate all of the available evidence for your question. 

• Exclusive to medicine or healthcare: Systematic reviews are commissioned or developed for many different disciplines interested in making evidence-informed decisions. 
• Exclusive to quantitative research: While many systematic reviews analyze studies that created quantitative data, qualitative and mixed methods reviews do exist and follow the systematic review methodology. 
• The only “good” review type: Different types of reviews are helpful depending on your context and question. Meet with a librarian to discuss if a systematic review approach is right for your team. 
• Exhausting (hopefully!): While systematic reviews can be time-intensive, the guiding methodology and opportunity for collaboration is invigorating to many researchers at all career levels. 

*For more information how to conduct a narrative literature review see: https://medlib.belmont.edu/LitReviews  

One of the nice things about systematic reviews is that there are well-established standards and guidelines for this research methodology.  

JBI Manual for Evidence Synthesis  

• guides authors who wish to conduct systematic and scoping reviews following JBI methodologies 
• Each chapter is devoted to the synthesis of different types of evidence to address different types of clinical and policy-related questions. 
• Easy to apply, step-by-step guidance 

Cochrane Handbook for Systematic Reviews of Interventions 

• The official guide that describes preparing and maintaining Cochrane systematic reviews 
• includes guidance on the standard methods applicable to every review (planning a review, searching and selecting studies, data collection, risk of bias assessment, statistical analysis, GRADE and interpreting results) 

Reporting Guidelines 

• PRISMA-Checklist and Flow Diagram 
• PRISMA-P extension for protocols 
• PRISMA-S extension for search reporting 

While systematic reviews are generally considered to be at the top of the “pyramid of evidence,” not all systematic reviews are created equal. Like any other research, they should be discussed, critiqued, and updated. Here are some tools for appraising systematic reviews: 

Critical Appraisal Skills Programme Systematic Review Checklists (CASP) 

• Use these checklists to analyze how a systematic review was conducted. 

Risk of Bias Assessment Tool for Systematic Reviews (ROBIS) 

• Use this tool to (1) assess relevance (optional), (2) identify concerns with the review process and (3) judge risk of bias for systematic reviews. 

• Grant, M.J., & Booth, A. (2009). A typology of reviews: an analysis of 14 different review types and associated methodologies. Health Information and Libraries Journal, 26: 91-108. DOI: 10.1111/j.1471-1842.2009.00848.x 
• Munn, Z., Stern, C., Aromataris, E. et al. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol 18, 5 (2018). https://doi.org/10.1186/s12874-017-0468-4 
• Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual review of psychology, 70(1), 747-770. 
• Allen, C., Walker, A. M., Premji, Z. A., Beauchemin-Turcotte, M. E., Wong, J., Soh, S., Hawboldt, G. S., Shinkaruk, K. S., & Archer, D. P. (2022). Preventing persistent postsurgical pain: A systematic review and component network meta-analysis. European journal of pain (London, England), 26(4), 771–785. https://doi.org/10.1002/ejp.1915  
• Hodder, R. K., O'Brien, K. M., Wyse, R. J., Tzelepis, F., Yoong, S., Stacey, F. G., & Wolfenden, L. (2024). Interventions for increasing fruit and vegetable consumption in children aged five years and under. The Cochrane database of systematic reviews, 9(9), CD008552. https://doi.org/10.1002/14651858.CD008552.pub8   
• Jefferies, D., McNally, S., Roberts, K., Wallace, A., Stunden, A., D'Souza, S., & Glew, P. (2018). The importance of academic literacy for undergraduate nursing students and its relationship to future professional clinical practice: A systematic review. Nurse education today, 60, 84–91. https://doi.org/10.1016/j.nedt.2017.09.020