Evaluation Principles

Evaluation Process

 

Our vision is a nation that has solved the disease of addiction to alcohol and drugs. We understand that “solving addiction” – arguably our nation’s most critical public health and social problem - is a complex and challenging task. Our evaluation process is designed to help us understand where our organization is today in moving toward that vision, including where we’re succeeding and where we’re falling short.

As Jim Collins describes in his book, Good to Great, great companies pay ruthless attention to their reality as they’re on their path to attaining their vision. Our evaluation process is based on this notion. We keep our eye on our vision, but carry forward with an unrelenting focus on data so we have an ongoing, honest assessment of the impact of our work.

 
Developmental Evaluation

Traditional evaluation (formative and summative) focuses on the carrying out of a prescribed intervention, making improvements and evidenced by the results of data analysis. The goal is to validate the intervention so it can be widely replicated.

Developmental evaluation is an agile and evolving process. It’s specifically designed to evaluate an innovation of a model or intervention within a highly complex, dynamic environment. We utilize a developmental evaluation approach because it’s a powerful method to support social innovation.

Developmental evaluation “is grounded in systems thinking and supports innovation by collecting and analyzing real-time data in ways that lead to informed and ongoing decision making as part of the design, development, and implementation process.” 

Our commitment to high-quality evaluation means that we have an internal evaluator on staff. Our evaluator works in close partnership with the innovators on our team to ensure our program development is fully data informed.

By focusing on understanding what’s happening as a new approach is implemented, development evaluation can help answer questions such as:

  • What is emerging as the innovation takes shape?
  • What do initial results reveal about expected progress?
  • What variations in effects are we seeing?
  • How have different values, perspectives, and relationships influenced the innovation and its outcomes?
  • How is the larger system or environment responding to the innovation?

The evaluator is right at the table with the social innovators on our team to ensure we get the highest level of accountability and fidelity to our mission and vision. With this approach, we ensure we’re consistently questioning our work:

  • Are we doing what we say we are doing?
  • Are we being true to our vision?
  • Are we looking at reality with our data and is that connecting directly to our vision?
  • What are we observing that’s emerging or different and what interventions will be put in place because of that?
 
Continuous Learning

Our team incorporates evaluation insights in developing and testing new hypotheses while acknowledging the reality of change in our complex environment.  We know there’s a high probability of encountering unexpected impacts and outcomes of the work that we do. The agility we have with our evaluation process allows us to be intentional in the use of data in all aspects of our decision making and routinely evolving our work as a result.

 
System-Wide Focus

In addition to focusing on the outcomes related to the clients we serve, our evaluation model is also designed to question the complex systems (healthcare, criminal justice, social services, etc.) surrounding the disease of addiction.

We use a double-loop learning process, which means that we don't stop at examining our peer-to-peer recovery coaching model. Instead we take that next step in understanding the complexities and brokenness of the community-wide system so we can change everything about it.

 

Key Evaluation Principles:

 
  1. Follow the guiding principles of evaluation as set forth by the American Evaluation Association.
  2. Involve stakeholder voices in the design, evaluation and reporting process. 
  3. Adapt to recognize and embrace the innovative nature of the work that we do.
  4. Embrace and represent the complexity of the disease of addiction; of the impact of the disease on the person, families and communities; and of the systems interactions around the disease.
  5. Provide timely, and often rapid, feedback to the innovation team on the results of the ongoing data collection process.