Frequently Asked Questions
Here you will find answers to many of the most frequently asked questions about our approach to evaluation of whole-system, place-based approaches to physical activity inequalities. If you can’t find the answer to your question here, please contact us using the form at the bottom of the page.
Theory of Change
Why is it important to develop a theory of change?
A Theory of Change can help Place Partners work through their local place-based systemic approach. It can act as a valuable tool to guide the cycle of developing, embedding, and evaluating and learning about a place-based systemic approach. It involves thinking through (theorising) and making explicit how and why change in outcomes is envisaged to occur, informed by a deep understanding of the context (circumstances within a Place). A Theory of Change can also help to clarify the assumptions held between colleagues and partners about what is needed, and why. This can positively contribute to building a shared understanding between partners, and enhance the planning and implementation of the approach taken forward.
How do you develop a theory of change?
A Theory of Change is as only as good as the process you go through to develop it. The best way to ensure it represents local circumstances and priorities is to include partners from communities and across sectors when discussing and deciding on the intended outcomes, and the pathways to achieve them.
See part 3 of our Theory of Change Descriptor for more details.
Place Evaluation and Learning Approach
What do we mean by a whole systems approach?
A whole systems approach (WSA) for physical activity is described in our forthcoming paper (Shearn et al., forthcoming) as, “Action to address barriers and enablers to physical activity at a combination of levels including the social/structural drivers of physical activity levels. Practically this may mean working public bodies such as local authorities and healthcare, businesses and the VCSFE sectors alongside residents and community members on strengths and remove barriers to enable people to move more in everyday life. It may also demand action on a range of structural, policy and commercial drivers that impinge from outside a locality. A WSA also recognises the intersections and interactivity both between these courses of action and the active influence of the context in which they are situated.”
Configurational Comparative Analysis (CCA)
What is the CCA survey?
CCA is a research method that integrates both qualitative and quantitative data to compare patterns across case-studies and identify which conditions, or combinations of conditions, lead to a desired outcome. The NELP team use the survey to collect data about the work taking place in local Places, as part of its evaluation of place-based systemic approaches to address physical activity inequalities across England.
Further information on CCA is available here , including the survey guidance.
Will I have to complete the survey in one go?
Each Place will have a unique survey link that will save your progress as you complete the survey. This allows you to close the survey and return to it at a later point without losing the information submitted.
Can more than one person in a Place fill in the survey?
We recommend one/two people completes the final survey submission. You will be provided with a survey link that is unique to your Place. However, your response should be based on input from stakeholders across the Place.
Should we include data from a range of stakeholders in our completed survey?
Before you complete the survey, we recommend holding a session with a range of stakeholders from your Place to collate data/examples to help you complete the survey. We do not recommend a specific number of people; we advocate a range of perspectives and areas of work. Further recommendations and resources for engaging stakeholders in the process are included in the CCA survey guidance.
Who am I completing the survey for – what counts as a Place?
For consistency from all submissions, we consider each Place as the Local Authority area it represents. Therefore, if you are representing multiple Places, you should complete 1 survey per Place. For example, if an Active Partnership represent 3 different Places, they will submit 3 separate survey responses. Each Place will be given a unique survey link to complete.
How much information/ how many examples do I need to include?
We recommend capturing examples from a range of stakeholders in Place to capture diverse viewpoints, along with a brief explanation of your reasoning for your maturity assessment, including consideration of contextual conditions as well as the activity taking place in this field. Contextual conditions may relate to the demographics and histories of the Place, and/or the organisational relationships and contexts within which this work is taking place. We do not need detailed descriptions of the examples used, but rather a brief explanation of what is happening or has happened, and how this relates to the particular system maturity condition. Your examples may come from, community organisations, operational teams, and senior leaders in your Place, and one example may well relate to more than one condition.
What should we do if there is no agreement of the maturity rating between stakeholders?
The maturity rating should reflect the overall Place, so it is important that examples are collected from a range of stakeholders in the Place. The maturity rating should reflect the data and examples gathered during this process so if there is not a clear agreement on the scoring it may be beneficial for a few people with detailed understanding of the work to review the data available and decide on an appropriate rating. The NELP team will also look across cases and moderate the ratings based on a range of information and examples.
Can we collect and submit responses in another format?
Some people may find it easier to capture and collate information in another format such as Word or Excel, and that’s fine, but we would ask you to copy the information into the online survey response to enable use to process it effectively. If you have any technical difficulties with using the online survey, please contact us by email: nelp@shu.ac.uk
System Maturity Matrix
What is the System Maturity Matrix for?
The System Maturity Matrix (SMM) has been developed by NELP in collaboration with Place partners and Sport England, as part of our conceptual model for the evaluation of Sport England funded Place-based approaches. It draws on the evaluation and learning undertaken with the Places as well as the wider literature. The summer 2026 version has been refined in response to feedback from Place partnerships and Sport England, as well as the reflections of the NELP team. The SMM provides a framework for thinking about aspects of the Place and the work taking place (Cross-cutting Conditions) relevant to the PBSA. It describes different conditions and sub-conditions that may have effects on physical activity inequalities (see below). The Place is defined by the Local Authority boundary for these purposes.
The Matrix is intended to help facilitate and prompt reflection and focussed discussions among stakeholders in the Place, including partner organisations and communities. For local programme teams and partners, this process can prompt reflection and discussion, leading to shared understandings of challenges and informing local priorities and actions. It also enables NELP to undertake Configurational Comparative Analysis (CCA), which identifies patterns of maturity and associated outcomes across different Places. This will help us to collectively consider how and why intended change is, or is not, happening.
Do I need to complete all the System Maturity conditions?
We ask you to submit a response for all conditions, but we recognise that some conditions may be less relevant or less well developed in your Place and therefore have less examples/data available than others. Please include an explanation for your assessment for each condition, supported where possible with examples, to enable the NELP team to understand your survey response.
EvalC3
What is the Eval C3 tool? Why do we use it?
EvalC3 is a tool developed by Rick Davies for developing, exploring and evaluating predictive models of expected outcomes. It was originally developed as an Excel app, designed for use in the monitoring and evaluation of the achievements of development aid projects (and parts thereof). However, it also has much wider applicability and is now available as an online tool via the link on this website.
EvalC3 enables users:
- To identify sub-sets of attributes that describe an intervention & its context, and which are good predictors of the achievement of an outcome of interest.
- To compare and evaluate the performance of these predictive models.
- To identify relevant cases for follow-up within-case investigations to uncover any causal mechanisms at work.
To find out more about Eval 3 – go to the subpage.
Realist Evaluation
What do we mean by realist evaluation?
Realist evaluation was initially developed in Pawson and Tilley’s seminal work, Realistic Evaluation (1997), and has been described by Westhorpe et al. (2011):
“Realist Evaluation changes the basic evaluation question. It does not ask what works or does this work. It asks ‘what works for whom, in what contexts, in what respects and how'. A realist approach assumes that programmes are ‘theories incarnate’. That is, whenever a program is implemented, it is testing a theory about what ‘might cause change’, even though that theory may not be explicit. One of the tasks of realist evaluation is therefore to make the theories within a programme explicit, by developing clear hypotheses about how, and for whom, programmes might ‘work’. The implementation of the programme, and the evaluation of it, then tests those hypotheses. This means collecting data, not just about programme impacts, or the process of programme implementation, but also about the specific aspects of programme context that might impact on programme and about the specific mechanisms that might be creating change.”
Conditions
Is there a particular condition that carries more importance than others, or are all conditions intended to have equal weight?
The conditions are probably best understood as interconnected rather than equally weighted. Different conditions appear to play different roles within a system, and some seem to act more as enabling conditions in practice. For example, work across NELP suggests that community perspectives, supported community-led action, enabling processes, and VCSE capacity often appear repeatedly in places demonstrating broader systems maturity. This does not mean they are “more important,” but that they may help create conditions in which other forms of change become easier. Rather than a hierarchy, the conditions appear to function as a set of interacting and reinforcing relationships.
Do all nine conditions need to be in place for a piece of work to be considered successful? or do we just need to move toward them?
No, progress can clearly occur through focusing on conditions. A single project or intervention is unlikely to demonstrate all nine conditions simultaneously. However, places showing stronger systems maturity over time often demonstrate increasing alignment across multiple conditions rather than isolated progress in only one area.
This distinction between project success and wider system maturity is important. At present, the framework appears most useful as a way of understanding movement and progression rather than as a checklist requiring full completion.
Place-based systems are dynamic and uneven. Conditions may strengthen or weaken over time, and maturity is unlikely to look identical across all places. Demonstrating reflection, adaptation, alignment, and progression may therefore be more realistic than expecting every condition to become fully embedded in the same way everywhere.
- Are multiple conditions required simultaneously for a single piece of work to succeed?
Often, yes. Even where one condition is the main focus of activity, wider conditions frequently shape whether change becomes sustainable. For example, physical environment work may depend heavily on collaboration, community trust, governance arrangements, and inclusive engagement.
This reflects the wider systems principle that outcomes emerge through interactions between conditions rather than through isolated interventions alone.
- Does application of the conditions vary depending on the starting point of a place?
Yes, and this is probably essential to acknowledge. Places begin from very different histories, capacities, relationships, and inequalities. As a result, progression will look different across places, and the same condition may manifest differently depending on local context. The SMM is therefore probably strongest when used to understand progression within a place over time rather than as a comparative ranking tool between places.
Different organisations bring different strengths and constraints. Local authorities may have stronger strategic and governance infrastructure, while VCSE organisations may have stronger community relationships and flexibility.
This reinforces the importance of interpreting conditions contextually rather than expecting them to look identical everywhere.
How is the “community-led” condition different from broader organisational collaboration?
This distinction is important. Organisational collaboration can still remain institutionally controlled, whereas community-led approaches imply stronger community influence over priorities, action, learning, or decision-making. The difference is therefore less about participation alone and more about power, ownership, and whose knowledge shapes the work.
How can conditions of change be better connected to real examples from the delivery plan?
This is probably where some of the strongest opportunities now sit. The modelling and evidence work already provides opportunities to show how conditions interact in practice, how progression occurs over time, and where tensions or trade-offs emerge. The most useful examples may not always be straightforward success stories, but examples of adaptation, partial progress, or challenges that reveal how systems change actually happens in practice.
