What Does “Pre-Appraised Evidence” Mean in Evidence-Based Practice?
When looking for evidence – where should you start? Well at the top of course! At the top of the hierarchy of evidence quality is pre-appraised evidence (Haynes, 2007).
If you go through your university or medical library, there are multiple databases to search, such as MEDLINE or CINAHL. You can also search from home using PUBMED. But as a busy clinician needing answers NOW, you don’t want to search databases that may have you wading through hundreds, if not thousands, of search results. Nor do you have the time to critically appraise the reports of primary or original research studies you find. To make your search process more efficient, you want to search pre-appraised evidence first!
What is Pre-Appraised Evidence?
When I say the evidence source contains pre-appraised evidence that means that someone else (an expert) has critically appraised the studies contained in the database for you and put them in a summary format. Summary literature, also called secondary literature, includes systematic reviews, meta-analyses, clinical practice guidelines, and critically appraised topics (CATs). Evidence-based practice summaries are also found in the form of online textbooks. These resources may be accessed for free or by individual or institutional subscriptions.
Pre-appraised evidence, also known as pre-filtered evidence or summarized evidence, is collected in databases – for example, rigorously conducted systematic reviews (SR) and meta-analyses (MA) can be found in the Cochrane Collaboration Library.
Pre-appraised evidence that is accessed from a credible source can be trusted by the clinician to have undergone a standardized process of evaluation.
Trusted evidence will save the clinician time because the source is updated with new evidence regularly and the appraisal work has already been done. In other words, you can count on the fact that the authors of this pre-appraised work have been rigorous in its development.
For sources that are not pre-appraised, YOU will have to determine whether the source can be trusted. So you will have to critically appraise any original research study you find because you want to make sure that you can believe the research findings. The same with other sources of evidence you use. For example, if you are using data from internal chart audits, you want to know how those data were collected and what oversight was used to ensure accurate data entry; and then how the data were analyzed.
Summaries, Syntheses, and Synopses
Summaries, syntheses, and synopses are considered pre-appraised evidence. These products are secondary analyses of primary research studies (AKA original research) and systematic reviews. In a secondary analysis, the researcher(s) analyzes published research studies to examine the data more closely or answer a research question different from the one asked in the primary studies. The secondary analysis can be of one research study or of many. A secondary analysis is a research method and the final product may be in the form of a synopsis, synthesis, or a summary.
A synopsis is a brief summary or overview of an original research study or of a systematic review. Synopses give the clinician a rapid read of the original research article (i.e., a critically appraised topic), of multiple research studies on a specific topic (synopses of studies), or of systematic reviews (synopses of syntheses), along with a commentary of the rigor of the information.
Synthesis is a process of taking multiple components and combining them into a new product. A synthesis, in evidence-based practice, is a systematic, comprehensive, critical review of high-quality research studies on a specific clinical question that then compiles the data from those studies into an “answer” to the clinical question posed. A systematic review, meta-analysis, and rapid review (Munn, Lockwood, & Moola, 2015) are types of syntheses.
A summary encompasses all the “lower” levels of evidence (i.e., original research, systematic reviews, and synopses) to produce the best evidence for clinical practice, such as a clinical practice guideline (DiCenso, Bayley, & Haynes, 2009). A summary ideally contains evidence from more than one systematic review (Munn et al., 2015).
All Evidence is NOT Created Equal!
A common misconception is that if a research study is published, it must be excellent. But that is false. Just because a research study is published doesn’t mean that the methods the researchers used are valid.
Having someone else do the work of critical appraisal is great, especially for busy clinicians, but before you trust these sources, be vigilant! I don’t want you to think that just because you find a systematic review that it is automatically good! Anyone can conduct and publish a systematic review!
Also, don’t make assumptions that because a synopsis, synthesis, or summary comes from a professional organization or national group (e.g., American Heart Association, National Kidney Foundation, etc.) that the document was assembled with the highest standards — unless you vet their process first!
Haynes (2007) warned that many sources claim to be evidence-based, but are not. The clinician should assess any source claiming to be evidence-based for the “underlying methods of assembly and assure [yourself] that these methods are sound. At each level [of evidence sources], the standards for evidence generation, retrieval, selection, and analysis should be explicit and at the highest evidence standard possible” (Haynes, 2007, p. 6).
Look for information on the organization websites or in the specific secondary study for how the reviews were developed. Once satisfied that the process is rigorous, then you can feel confident in the results of the review without having to get out your appraisal worksheets.
Pre-Appraised Evidence Databases
Point-of-care resources are vital to contemporary evidence-based practice. These resources allow you to quickly find evidence summaries on-the-go from your smartphone or tablet device.
There are a number of credible databases for finding pre-appraised evidence. Some of the databases are free and others are only available through a subscription. Check the databases available to you in your medical or university library.
Summary Evidence: Online, evidence-based textbooks or summary services include Clinical Evidence, Physician’s Information and Education Resource (PIER), UpToDate, MDConsult, First Consult, Nursing Consult, DynaMed Plus, Best Evidence Topics, and Essential Evidence Plus.
Clinical practice guidelines (CPGs) are evidence sources that fit in this layer, also. One of the best-known sources of CPGs is the National Guidelines Clearinghouse (www.guidelines.gov). Check out my post on EBP Resources specific to CPG sites for a list of helpful databases.
Synthesis Evidence: The Cochrane Library should be your FIRST stop to find summary evidence, in the form of systematic reviews, for clinical questions about clinical treatment. Evidence from the Cochrane Collaboration is considered the gold standard “for high quality, trusted information.” This is where you want to look first for systematic reviews about clinical treatment! Many libraries also provide access to Cochrane.
The DARE database (Database of Abstracts of Reviews of Effects) is a resource for synopses of syntheses and can be found in the Cochrane Library. For syntheses on education and other social topics, you can search the Campbell Collaboration.
Synopsis Evidence: There are a variety of journals that are known for summarizing high-quality original research evidence and clinical topics. Evidence-based abstraction journals that provide synopses include ACP Journal Club, Evidence-Based Mental Health, and Evidence-Based Nursing. Again, many of the evidence-based summary services noted above provide synopses also.
How to Quickly Determine Validity of Pre-Appraised Evidence
I will provide future blog posts on how to critically appraise different studies, but for now, here are some quick tips.
In a synopsis, synthesis, or summary, the research studies included in those publications would have been already appraised by the author. So you won’t have to appraise each study they included.
What you want to know is HOW the authors assembled their synopsis, synthesis, or summary.
What was the methodology used to search for the included studies, determine the methodological quality of each included study, and how did the author even choose the studies for inclusion? The author needs to tell you exactly what they did in the introduction section of their research report – that’s what is meant by transparency. You need to know the methods so that you can decide if you can believe the results.
There are groups and organizations that mandate a rigorous process for doing this kind of work. That kind of standardization is gold because once you understand the process used, you shouldn’t have to reappraise the methodology of every publication you read using that systematic standardized methodology.
For example, when the American Heart Association (AHA) puts out new or updated clinical practice guidelines, they are clear about how the working group assembled the guidelines at the beginning of the document.
I’ve read the standardized methodology for systematic reviews published on the Cochrane Collaboration library website. All authors writing for the Cochrane Collaboration follow their guidelines for writing a systematic review. When I read a systematic review from that site, I know the methods are rigorous, I can see how the author(s) reported their methods, and, therefore, have confidence in the reported results.
Because the processes outlined by these organizations are transparent, I don’t go through the work of appraising every AHA clinical practice guideline or Cochrane Review I read!
Bottom Line: If you find the synopsis, synthesis, or summary from a credible source, then the hard work of critical appraisal is done for you and you can use the results without a further methods critique on your part. But if the synthesis is done by a single author or a group of authors not associated with known credible databases, you will still have to determine how rigorous their methods were before you want to rely on their research results.
DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. Evidence Based Nursing, 12(4), 99-101. doi:10.1136/ebn.12.4.99-b
Haynes, R. B. (2007). Of studies, syntheses, synopses, summaries, and systems: The “5S” evolution of information services for evidence-based health care decisions. Evidence-Based Nursing, 10(1), 6-7.
Munn, Z., Lockwood, C., & Moola, S. (2015). The development and use of evidence summaries for point of care information systems: A streamlined rapid review approach. Worldviews on Evidence-Based Nursing, 12(3), 131-138. doi 10.1111/wvn.12094