Wednesday, 18 September 2019

Evaluation Tips Series;

This is part three in our evaluation tips series, you can see part one here and part two here.

Once you have decided on your demographics, it's time to look at the specifics of what it is you are trying to change or measure.

Are you trying to make people more connected? Less lonely? Then you can keep it simple and ask people how often they go out, repeat this at the end and see if there is an average change – or an individual change. Or you could ask do you have enough friends?

If your project or activity will have an impact on people's emotions and well-being, the Warwick Edinburgh Scale of Wellbeing is a really useful tool. You can see more about this here. 

A number of projects recently have been helping people back to employment. You can simply measure if people do get a job or go back into training or into volunteering. Often, though, this misses the majority of the journey. People who have never worked, might take a long time and need a lot of support to get back to employment. They might not get a job during the majority of the project, but you have made a difference in their live. In order to show this effectively, we have created a journey back to work road-map. It shows all the steps people may take on their journey from not working, to building confidence and skills.

You might want to measure if people had fun, if they moved more or were more physically active. Did they visit the doctor less or use less medication or support services? Did you change their attitudes or did they learn something new? For example, in a recent project about suffragettes we asked people to name any prominent suffragettes at the beginning and then again at the end and showed how many more people could talk about on a graph.

You might want to understand people's motivation and it can be helpful to find out why people wanted to be involved, and what attracted them. If you ask enough people too, what they think might motivate others, or what might make a better project, you can use this information to improve your services. These kind of questions do need to be specific, you can’t just ask, ‘what motivates you to do this?’ It's much better to ask something like ‘Why do you think people volunteer?’, and give five answers to choose from and an ‘other’ option. 

Some good questions to ask include:

  • Did you enjoy yourself? 
  • Would you recommend this to a friend? 
  • How often were you active in the last two weeks? (Ask at the start and again at the end and see what the change is).
  • How often did you visit the GP in the last three months? (Ask at the start and again at the end and see what the change is). 
  • Are you content with your employment status? 
  • Did you make friends? 
  • Do you leave the house as much as you want to? 
  • Does your health stop you from doing everything you want to? 

Don’t forget the staff! 
Staff, Volunteers, Artists and Providers can all be a useful source of information. You could get staff to assess people at the start and again at the end or you could ask staff what they thought worked well and what would they change?

Example; questions about marketing:
  • How did you hear about the project / activity?
  • Did you tell a friend?
  • Did you mention this on social media?
  • How should we share this?
  • Where should we send a press release?
    • Or ask staff: What marketing method was most successful?

These are just a sample of the kind of questions you might want to ask. These are questions which are mostly used with adults. We will talk about children and young people in another article. 

If you are in doubt; or feeling unsure about what to ask... here are some core questions. Asking just these will give you a really strong basic evaluation.
  • What did you enjoy the most? 
    • Or Staff: What worked best?
  • What did you enjoy the least?
    • Or Staff: What worked the least?
  • What could we do better?
    • Or Staff: What do you think could have gone better?
  • Would you recommend to a friend? 
    • Or Staff: Would you run this again?

Above all, keep it simple. People are more likely to understand simple questions and that makes them more likely to answer. We will talk about ways to collect information in part four. Impact is often all about ‘what changed’  so don't forget to ask something at the start, repeat it at the end and see what the change is.