Energise Manchester: How to Evaluate Behaviour Change, with Shortwork

by Blog

A guest post by Sandy Rushton, People Powered Retrofit. Originally featured on Medium.

On the Energise Manchester project, we’re asking a number of different questions about behaviour change and energy advice:

  • What behaviours do we want to affect on our project?
  • What are the capabilities, opportunities, and motivations that impact these target behaviours?
  • What interventions and activities will best affect behaviour change?
  • After we’ve done an intervention, did we achieve the target behaviour change?
  • How can the behaviour change methods used in this project help the team to deliver more effective community-based energy advice?

These questions, alongside requirements from our funders, result in a complex and diverse needs for evaluation.

Working with Shortwork to evaluate behaviour change

Shortwork is a social research organisation with expertise in qualitative and creative research. On the Energise Manchester project, they are responsible for project evaluation.

Rowena Hay held a workshop with members of the Energise Manchester approach in October 2023. The results of this workshop inform our project-wide evaluation approach. We worked together to create a visual representation of the different elements of our project using collage.

We defined our project’s:

  • Inputs: The resources invested into the project.
  • Outputs: The services or products being delivered on the project.
  • Outcomes: The measurable changes that our project makes.
  • Impacts: The project’s long-term contributions to the communities we work with (and beyond).

The resulting collage is displayed in the Carbon Co-op office, and Rowena created a digital version using Miro to keep as a reference point for the project.

Developing a logic model

From the project inputs, outputs, outcomes and impacts — as well as the Behaviour Change workshops run by Snook and ongoing input from the project delivery teams – Rowena developed an evaluation logic model for the Energise Manchester project.

Using the logic model, Rowena developed detail on the necessary data to be collected and methods to be used for collection for each intervention that will be delivered in the Energise Manchester programme.

The model lists relevant project outputs and outcomes for each intervention. Each output and outcome is linked to an Indicator (what we will measure) and Monitoring and Evaluation method (how we will collect the data we need). Target behaviours are starred, integrating the decisions on key target behaviours from prioritisation activities in Behaviour Change workshops.

From this, Rowena has also developed a toolkit of evaluation resources to support the project team in capturing this data in an appropriate, creative, and accessible way.

Developing hypotheses

In tandem with Rowena’s work on developing a logic model, Chandni Patel (Snook) developed a format for hypothesis statements for each behaviour of interest in the project.

This hypothesis statement format is a complementary approach to the logic model but presents the information in a narrative format.

A hypothesis statement reads like this:

Here is an example of the hypothesis statement for one of the target behaviours for our project’s community hub-based workshops, which you can also see outlined in the logic model above.

The narrative form of these statements helps delivery teams to quickly understand what behaviour they are affecting with their work, what data they need to collect, and why.

These statements will also be used as part of regular ‘wash-ups’, where the teams working on a particular intervention will review activities to date and reflect on what is working as expected and what is different or needs to be improved.

Summary

The behaviours of interest for the Energise Manchester project are many and varied. Rowena and Chandni’s work on evaluation will help our team to develop existing and new interventions in an informed way.

With the support of a toolkit of resources to aid data collection, the project team can get input from users of our service to iteratively improve the energy advice provision for the communities we work with on the Energise Manchester project.