Using literature reviews to develop evidence-informed programs

Recently, a leader of a Sonoma Valley nonprofit organization identified a serious community problem that her organization has the ability to tackle. Then, she immediately suggested that she and her staff could create a program from scratch to help solve this problem, which is not an uncommon response at all. However, a far better way to implement a solution is to conduct a little online research to see what programs may already have been conducted in similar communities with similar populations to address this same community need.

Through some online research, nonprofit leaders can find a literature review, also called a meta analysis, that identifies what is known and not known about a specific topic as a first step. According to Cristin Felso, executive director of Teen Services Sonoma, “Before beginning a new program or adding services to an existing program, it is a good idea to see if there is literature available that supports the perceived value/effectiveness of the proposed program or service concept.”

According to the Sonoma County Upstream Investments website (upstreaminvestments.org/), a literature review is a summary and synthesis of current and credible literature on a specific topic, issue, or practice method. It is a survey of scholarly articles, books and other sources that are relevant to the subject. A strong literature review includes: a clear statement of the subject of interest; a review of a credible number of studies about the topic; and a summary of the most influential, current, and frequently cited sources related to the topic.

Good literature reviews reference at least 25 studies, with many including over 100 studies. In addition, it is wise to use a literature review that has been written in the past 15 years, although some of the studies referenced in the review may be older than 15 years old, which is fine.

Although it is certainly possible to write a literature review, it is far easier to find one online that has already been developed and published by researchers who devote their careers to studying and evaluating the topic. According to Theresa Rhodes, Sonoma Valley Education Foundation program and grants manager, “Finding high-quality literature reviews which cover relevant, well-documented and current research confirms that the programs we support include best practices to help ensure positive outcomes.”

A good way to find the perfect literature review is to go beyond a regular Google search and use a Google Scholar search instead. Search using words associated with the topic of interest plus the words “literature review” or “meta analysis.” Recent literature reviews describing the outcomes that can be expected through the types of nonprofit programming being implemented in Sonoma Valley include the following:

• Youth who participate in ready-to-work programs (like those offered by Teen Services Sonoma) are more likely to be employed and less likely to live in poverty when they become adults.

• Financial education programs for Latinos (like those offered by La Luz Center) increase the financial knowledge of Latinos and cause positive behavioral changes, making them more likely to participate in home ownership, entrepreneurial practices, and other forms of financial asset development.

• Youth in formal, high quality relationships with adults (like those provided through Sonoma Valley Mentoring Alliance) attend school more regularly, have better attitudes and behaviors at school, and are more likely to pursue post-secondary education. Mentored youth are 46 percent less likely to initiate drug use; 27 percent less likely to initiate alcohol use; 52 percent less likely to skip school; and 32 percent less likely to hit someone than un-mentored youth.

• Listening to music, drawing pictures and making things to remember and honor their loved ones, staying busy, going to a grief group, and talking to other grieving children greatly benefits grieving children (like the type of programming offered by Social Advocates for Youth/WillMar).

Felso was delighted to discover through a literature review that the Ready to Work program offered by Teen Services Sonoma “makes it even more effective than other types of work readiness programs.” She added, “Having solid literature reviews to back an organization’s programs and services makes a real difference in applying for funding. Now we have experienced an increase in the number of grantors willing to fund the program, as well as an increase in the dollar amount received from past grantors.”

A comprehensive literature review can help an organization identify how to best evaluate its programs. Felso said, “After finding a literature review that showed that youth who participate in ready to work-style programs are less likely to engage in risky behaviors, we included this question in our pre- and post-program survey: ‘I am able to say no to things I know are wrong’ to be able to assess change in risky behaviors.”

The next time the leadership of a nonprofit organization wants to determine how best to solve a community problem, the organization and the community would be well served if the leadership identified what researchers have already discovered to be the best, evidence-informed outcomes, and then build on that knowledge.

B.J. Bischoff is the owner of Bischoff Performance Improvement Consulting, a Sonoma firm specializing in building the capacity of nonprofit organizations and public sector agencies to better serve their stakeholders. She assists her clients with strategic planning, training resulting in performance improvement, fund development, and community relations. She is Past President of Impact100 Sonoma and serves as a Sonoma County Board of Supervisors’ appointee to the Sonoma County Portfolio of Model Upstream Programs Review Committee. Contact her at [email protected].