A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993). The key characteristics of a systematic review are:
a clearly stated set of objectives with pre-defined eligibility criteria for studies;
an explicit, reproducible methodology;
a systematic search that attempts to identify all studies that would meet the eligibility criteria;
an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and
a systematic presentation, and synthesis, of the characteristics and findings of the included studies.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from http://handbook.cochrane.org.
"Systematic reviews should be undertaken by a team. [...] Working as a team not only spreads the effort, but ensures that tasks such as the selection of studies for eligibility, data extraction and rating the certainty of the evidence will be performed by at least two people independently, minimizing the likelihood of errors. First-time review authors are encouraged to work with others who are experienced in the process of systematic reviews and to attend relevant training." (Cochrane, Ch. 1)
"The review team should be capable of defining the clinical question and performing the technical aspects of the review. It should be multidisciplinary, with experts in SR methodology, including risk of bias, study design, and data analysis; librarians or information specialists trained in searching bibliographic databases for SRs; and clinical content experts." (IOM, Ch. 2)
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.
Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK209515/
Additional Reading:
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, 7(2), e012545. https://doi.org/10.1136/bmjopen-2016-012545
Your question should be not be too broad. You may want to do a quick scope of the literature to get a sense of how your question should be focused. You may consider using a Question Framework to help you define your question and focus your concepts.
Search the literature to determine if a review was recently published on this topic. If yes, a review may not be needed or your should consider changing your research question.
"Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review." (Moher)
"A protocol should be made publicly available at the start of an SR in order to prevent the effects of author bias, allow feedback at an early stage in the SR, and tell readers of the review about protocol changes that occur as the SR develops." (IOM)
Registering a protocol minimizes bias, increases transparency, and reduces duplication of effort. (PloS)
Visit the following registries for additional information or to publish a protocol:
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & PRISMA-P Group (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK209515/
The PLoS Medicine Editors (2011) Best Practice in Systematic Reviews: The Importance of Protocols and Registration. PLoS Med 8(2): e1001009. https://doi.org/10.1371/journal.pmed.1001009
"The overarching objective of the SR search for evidence is to identify all the studies (and all the relevant data from the studies) that may pertain to the research question and analytic framework. The task is a challenging one."
Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US). https://www.ncbi.nlm.nih.gov/books/NBK209515/
Additional Readings:
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & PRISMA-S Group (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617–626. https://doi.org/10.1016/j.jclinepi.2014.11.025
Selection of Evidence
"Experts agree that using two screeners can reduce error and subjectivity."
Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US).https://www.ncbi.nlm.nih.gov/books/NBK209515/
The PRISMA Flow Diagram details a two-step screening process:
Evaluation of Evidence
"There is good empirical evidence that particular features of the design, conduct and analysis of randomized trials lead to bias on average, and that some results of randomized trials are suppressed from dissemination because of their nature."
Types of bias can include:
Higgins J.P.T., Thomas J., Chandler J., Cumpston M., Li T., Page M.J., Welch V.A. (Eds.). (2020 update). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. Available from https://training.cochrane.org/handbook/current
Data Abstraction
Data abstraction is the systematic collection of data elements from included studies in a review. Data elements could include study design, intervention, outcomes measured, results, etc. This is best achieved through a standardized form derived from your research question and inclusion/exclusion criteria. Pilot the use of this form before full implementation. For additional help on data abstraction, visit the DSRS LibGuide.
Appraisal
Study appraisal systematically examines factors such as
Analysis and Interpretation of Evidence
"The purpose of standards for evidence synthesis and assessment - as in other SR methods - is to set performance expectations and to promote accountability for meeting those expectations without stifling innovation in methods. Thus, the emphasis is not on specifying preferred technical methods, but rather the building blocks that help ensure objectivity, transparency, and scientific rigor."
Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US).https://www.ncbi.nlm.nih.gov/books/NBK209515/
The synthesis of evidence includes a qualitative synthesis of included studies which describes study methodology, strengths and limitations, patterns across studies, potential bias in study design, and the relevance of studies to the populations, comparisons, cointerventions, settings, and outcomes or measures of interest.
The synthesis of evidence may also include a meta-analysis that pools data from included studies. A meta-analysis will address the heterogeneity among study effects, statistical uncertainty, and the sensitivity of conclusions to changes in the protocol, assumptions, and study selection (Institute of Medicine, 2011).
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an established guideline for reporting systematic reviews.
prisma-statement.org/
It is recommended that the PRISMA 2020 be referred to "early in the writing process, because prospective consideration of the items may help to ensure that all the items are addressed."
"PRISMA 2020 is not intended to guide systematic review conduct, for which comprehensive resources are available. However, familiarity with PRISMA 2020 is useful when planning and conducting systematic reviews to ensure that all recommended information is captured."
"The PRISMA 2020 statement (including the checklists, explanation and elaboration, and flow diagram) replaces the PRISMA 2009 statement, which should no longer be used [...] The PRISMA 2020 checklist includes seven sections with 27 items, some of which include sub-items (Table 1). A checklist for journal and conference abstracts for systematic reviews is included in PRISMA 2020. This abstract checklist is an update of the 2013 PRISMA for Abstracts statement,[54] reflecting new and modified content in PRISMA 2020 (Table 2). A template PRISMA flow diagram is provided, which can be modified depending on whether the systematic review is original or updated (Fig 1)."
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ (Clinical research ed.), 372, n71. https://doi.org/10.1136/bmj.n71
www.prisma-statement.org/Extensions/Searching
"Incomplete reporting of the literature search methods can introduce doubt and diminish trust in the final systematic review conclusions. If researchers are unable to understand or reproduce how information was gathered for a systematic review, they may suspect the authors of having introduced bias into their review by not conducting a thorough or pre-specified literature search. After observing the high number of systematic reviews with poorly reported literature searches, we sought to create an extension to the PRISMA statement. Our aims were four-fold:
To provide extensive guidance on reporting the literature search components of a systematic review.
To create a checklist that could be used by authors, editors, and peer reviewers to verify that each component of a search was completely reported and therefore reproducible.
To develop an interdisciplinary checklist applicable to all method-driven literature searches for evidence synthesis.
To complement the PRISMA Statement and its extensions."
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & PRISMA-S Group (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
Information on this page has been shared and adapted, with permission, from the "Planning for your literature review in the health professions" LibGuide created by Carrie Price at Albert S. Cook Library, Towson University, under CC BY-NC 4.0.