For funders seeking to make evidence-based investments, understanding what we know and do not know from existing research is an important first step. But the usual approach of systematic reviews and synthesizing research can be expensive and can take more than a year to complete. Furthermore, systematic reviews are often written in a manner only accessible to other researchers and may not address the questions of greatest relevance to a funder.
Between 2015 and 2017, the Robert Wood Johnson Foundation (RWJF) commissioned AcademyHealth, the professional society for health services research, and its Translation and Dissemination Institute, to develop and test a “rapid evidence review” (RER) process that could meet the needs of decisionmakers for fast, low-cost, but rigorous syntheses of evidence about health-related services and programs.
AcademyHealth developed its process by studying similar approaches used in other countries to support health policymaking (Hite 2016). RERs can be completed in time frames that range from a few days to a few months. Several features of the RER process are key to its feasibility: (1) availability of already completed systematic reviews, (2) input from subject matter experts who guide the search for evidence and review draft products, and (3) ongoing dialogue with the client to assure the end-product is useful (Gluck 2016). To make sure decisionmakers can quickly and easily digest the results, the final product begins with a one-page summary of bottom line findings with more detailed analysis presented in the pages that follow.
Originally envisioned as a way to help the public sector make health policy decisions, the Colorado Health Foundation (TCHF) and RWJF have used the RER process to inform internal discussions about potential investments and to assist potential grant applicants. Three examples of recent reviews highlight the potential value and limitations and provide insights into how RERs can be most useful for health funders.
What are effective strategies for recruiting and retaining rural primary care professionals?
This RER addressed a relatively narrow question for which there were a significant number of systematic and other types of relevant reviews. At the outset, TCHF asked that AcademyHealth exclude studies of telehealth and efforts to expand non-physicians’ scopes-of-practice. The RER, completed in six weeks, included 11 systematic reviews that pointed to consistent findings: some strategies are effective in recruiting providers to rural areas, but less so in retaining health professionals in these settings. While few approaches have a positive effect on both recruitment and retention, combining individual interventions may be the most effective way to expand and maintain the rural primary care workforce. The idea of combining strategies aligned with TCHF’s interests, and the foundation made the final review available as a resource for potential applicants (Gluck 2018).
What interventions help teens and young adults prevent and manage behavioral health challenges?
This request from TCHF came at an early point in the foundation’s thinking about potential funding on the topic. As a result, the question and relevant evidence were broad, covering a variety of approaches, interventions, behavioral health conditions, settings, and subpopulations. The findings were more general than those of the rural provider RER: targeted interventions focusing on resilience and/or cognitive behavioral therapy can lead to small, but positive improvements in a range of behavioral health outcomes for youth. This review, completed over six weeks, could serve as a starting point for those new to the topic, but its breadth means that additional efforts are needed to understand exactly what works for whom and under what circumstances (Smith 2018).
What opportunities exist to improve consumer engagement in health care?
RWJF wanted to understand the current state of the evidence, including information from the foundation’s own work, about two specific areas: (1) consumer engagement in clinical interactions and (2) consumers’ use of information to inform health care decisions. The foundation identified several specific questions of interest under these two broad categories. Because each question could generate its own RER, AcademyHealth organized its synthesis around findings from 36 systematic reviews relevant to the two broad topics. This RER served as background for a meeting of 30 experts and stakeholders convened by AcademyHealth and the foundation for a discussion grounded in the evidence, but further contextualized by practical application and emerging knowledge (Gerlach 2017).
Key Lessons
- RERs provide an efficient, timely way to infuse foundation strategy discussions with rigorous research findings. However, we found that RERs yield better results when the query is a straightforward, narrowly focused question rather than a broad topical area. More specific questions allow one to dive deeper into the available evidence, potentially bringing in research not in systematic reviews, evidence from the non-academic “grey literature,” experts, or tacit knowledge from the frontlines of service delivery. A key insight is that it appears easier to pose narrower questions in later planning stages when the focus of a program or strategy is more concrete.
- Conversely, RERs can be useful, though challenging to complete, when funders’ interests are broad—when they have multiple questions or interest in a diversity of approaches—and the time frame is short. This was the case in the consumer engagement and behavioral health RERs, respectively. RERs focused on broader topics will likely have higher level findings, perhaps constituting an environmental scan that assesses the state of available research broadly. A key insight is that broader reviews could be followed by more focused reviews as the funder’s interests narrow. Alternatively, they could be followed by direct experts’ and stakeholders’ input as a way to round out the evidence base with qualitative input.
- Key to all three of these reviews was regular communication between AcademyHealth and funders throughout the review process. This helped ensure the funder and AcademyHealth had similar expectations of the RER, and it allowed AcademyHealth to get guidance about what would be most useful to the funder as it learned more about the available research base. For example, TCHF and AcademyHealth were able to streamline the focus of the the rural recruitment and retention RER by excluding approaches TCHF was not interested in pursuing.
- So far, AcademyHealth has not conducted an RER for which relevant systematic reviews do not exist. However, many systematic reviews only include studies with the most rigorous designs, like randomized controlled trials, even though good studies using somewhat less rigorous approaches may be plentiful and similarly useful. With additional resources, including time, it can be possible to bring such research into the review. An important insight is that when systematic reviews come to clear conclusions, funders can have high confidence in the results. The downside is that useful relevant evidence may be left on the table.
Recognizing the strengths and limitations of this process is an important step for making RERs a useful resource for health funders seeking to make impactful, evidence-based investments.
References
Gerlach, Lauren and Michael Gluck, “What Opportunities Exist to Improve Consumer Engagement in Health Care?” Washington, DC: AcademyHealth. 2017.
Gluck, Michael. “Can Evidence Reviews Be Made More Responsive to Policymakers?” Poster. Washington, DC: AcademyHealth. 2016.
Gluck, Michael and Kyu Kang. “What Are Effective Strategies for Recruiting and Retaining Rural Primary Health Professionals?” Washington, DC: AcademyHealth. 2018.
Hite, Jared and Michael Gluck. “Rapid Evidence Reviews for Health Policy and Practice.” Washington, DC: AcademyHealth. 2016.
Smith, Samantha and Lauren Gerlach. “What Interventions Help Teens and Young Adults Prevent and Manage Behavioral Health Challenges?” Washington, DC: AcademyHealth. 2018.