Here is where you will find how to access and use tools for evaluation of whole-systems, place-based approaches.

evalc3-logo.svg

### Background

The design of EvalC3 Online makes use of two sets concepts and methods:

- Qualitative Comparative Analysis, a body of methods developed in Political Science. Especially its view of causality (equifinality, asymmetry, conjunctural) and the importance of combining cross-case and within-case analysis)
- Supervised machine learning, a body of methods used for the development of predictive models. Especially what is known about different search algorithms and how the performance of prediction models generated by these algorithms can be evaluated.

### Overall purpose

EvalC3 Online enables users:

1. To identify sub-sets of attributes that describe an intervention & its context, and which are good predictors of the achievement of an outcome of interest.
2. To compare and evaluate the performance of these predictive models.
3. To identify relevant cases for follow-up within-case investigations to uncover any causal mechanisms at work.

### Workflow

The worksflow diagram below shows the main sequence of steps that would normally be followed, when analysing a set of of data. Recursion and reiteration of earlier steps is also likely to be common. 

![Intro flow chart 5.png](https://res.cloudinary.com/dzhwmblon/image/upload/v1719161055/Intro_flow_chart_5_fdcd5cae9d.png)

### Range of uses

EvalC3 can be of use at all stages of a project cycle:

**During project selection**
- To identify what attributes of project proposals are the best predictors of whether a project will be chosen for funding, or not
- To identify how well a project proposal appraisal and screening process is as a predictor of the subsequent success of projects in achieving their objectives

**During project implementation**
- When the effectiveness of specific activities are being measured using survey instruments which include both specific (facet) and general (global) measures of satisfaction with service or product delivered. EvalC3 can identify what combination(s) of facets best predict global satisfaction. For example:
- Participants experiences with workshops and training events
- Donors and grantees experiences of their working relationships with each other

**During a project evaluation**
- “Causes of effects” analysis: To identify what combination(s) of project activities (and their contexts) were associated with a significant improvement in beneficiaries lives.
- “Effects of causes” analysis: To identify what combinations of improvements in beneficiaries lives were associated with a specific project activity (or combination of)
- To identify “positive deviants” – cases where success is being achieved despite the fact that failure is the most common outcome. 

**During a review of existing evaluations**
- Re-analysing data that was collected, to verify the results. This is often possible with QCA based evaluations because QCA data sets are usually published as annexes to evaluations
- Synthesising the results of multiple evaluations, into prediction rules concerning different types of outcomes

For more information on each of the stages in the flow chart above, see [How to use EvalC3](https://evaluatingcomplexity.org/resources/how-to-use-eval-c3-online)

### **Acknowledgements**

EvalC3 Online was built by [Reyt](https://www.reyt.co.uk/), based on the Excel app EvalC3, built by Mark Skipper at [Aptivate](http://aptivate.org/), as designed by [Dr Rick Davies](https://richardjdavies.wordpress.com/).

With funding provided by [Sports England](https://www.sportengland.org/), managed by a team led by Dr Katie Shearn at the [Sheffield Hallam University Advanced Wellbeing Research Center](https://www.shu.ac.uk/advanced-wellbeing-research-centre/research/transforming-places-for-health-and-wellbeing)


EvalC3 Online: Introduction

This publication is aimed at organisations, community groups, students and academics who wish to use Most Significant Change (MSC) to help monitor and evaluate their social change programs and projects, or to learn more about how it can be used. 

The technique is applicable in many different sectors, including agriculture, education and health, and especially in development programs and can be applied to be many different cultural contexts. 

The MSC technique is a form of participatory monitoring and evaluation. 
- It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing the data. 
- It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. 
- It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole. 

Essentially, the process involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by ‘searching’ for project impact and once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact.

Further information and guidance can be found [here](https://mande.co.uk/wp-content/uploads/2018/01/MSCGuide.pdf).

Tools

EvalC3 Online: Introduction

Most Significant Change Technique