Thursday, 8 August 2019

Evaluation Tips Series; What questions do you need answering?

This is part two in our Evaluation Tips Series, you can see part one here

Once you have planned out who the evaluation is for, it's time to think about what questions you need answering.



Key Questions to think about:

  • What are / were the project / activity aims?
    • If a project is trying to improve confidence or health, make sure you ask about changes in confidence or health
  • What is the impact you expect to happen?
    • For example, do you think people will be more physically active as a result of the work your charity does? Then ask them how many days they were active in the last two weeks at the start of your work with them and repeat this and see what has changed
  • What is happening during the project? 
    • Do people take part for years or on just one day? If it's just one day, you will need to collect all your information then, if it is years you might want to have an annual collection month and talk to all your participants every February for example
  • What do we want to know the answers to?
    • Sometimes it can be as simple as are the sessions on the right day and time, how did people hear about the programme and would they tell a friend to come along? Sometimes it's more nuanced and you want to know if they have learnt about a particular topic or that they get on better with their family because of the work you have done
  • How much time is there to collect?
    • If you only have a little bit of time, only ask a few questions
  • How much time is there to analyse?
    • If you only have an hour to analyse the information, just ask a few questions. It's better to look at what you learn rather than collect lots of information and have no time to do anything with it! 
  • What information already exists?
    • Often projects overlook the information they already collect, if there are registers - do people come regularly? If you have an enrolment form, you may already collect all the demographics you need. 



Demographics

Demographics simply relate to the kind of people you work with; young or old, male or female, whether from an ethnic minority or not. Funders almost always want to know if you have reached the people you thought you would so if your evaluation is designed for funders, you will almost certainly want to collect this information.

Collecting Age

Here you can choose whether to ask people their date of birth, or get them to tick age brackets. Some people do not like giving this information so always make sure there is a 'prefer not to say' option.

The reason for collecting age is simply to find out how old the people are who are taking part. Once you know the age you can answer the following questions; Is this project aimed at a particular age group? Did you reach them? Who is missing and what does that mean for planning in the future?

Collecting Gender

This is usually best kept simple and you can ask people to tick

  • Male
  • Female
  • Non Binary
  • Prefer not to say


Unless there is a reason otherwise, you would normally like to try and hit 50% male and 50% female. If you didn't make a 50/50 split, ask yourself why not? What does that mean?

Collecting Ethnicity

If you do only see people very briefly you could ask people if they consider themselves from a white background or from a black and minority ethnic background (often referred to as BME) or give people the option to choose 'prefer not to say'. Sometimes people refer to BAMER now which stands for black and minority and refugee backgrounds. On the whole, though, it's probably best to use the standard ethnicity question as recommended by the census, office for national statistics and almost all funders. The standard question is below, in italics, so you can just copy it from here if you want.



What is your ethnic group?
White
·         English/ Welsh/Scottish/ Northern Irish
·         Irish
·         Gypsy or Irish Traveller
·         Other White background
Mixed
·         White and Black Caribbean
·         White and Black African
·         White and Asian
·         Other Mixed/ multiple ethnic background
Asian or Asian British
·         Indian
·         Pakistani
·         Bangladesh
·         Chinese
·         Other Asian background
Black or Black British
·         African
·         Caribbean
·         Other Black/African/Caribbean background
Other
·         Arab
·         Other

·         Prefer not to say

Collecting Postcodes

Here at The Evaluator we always try and collect postcodes. Postcodes are very helpful if you want to show the location of people on a map – which can be done through Google Maps for free.

Collecting Disability

There are a few ways of collecting this but we tend to go for 'Do you consider yourself disabled?' and give four options;

  1. Yes, very
  2. Yes, a little
  3. No 
  4. Prefer not to say
That's because we believe in simple questions. It is really important to collect information on disability, we often find that people with disabilities don't have the same outcomes as people who are not disabled and this means projects need to think carefully about the work they are doing and how they can make it better, so everyone can achieve. 

Finally, some funding bodies want information collected on Socio-Economic Status – Employment Status or Household income, and Sexual Orientation and Gender Reassignment. If your project is funded, check your funding body requirements first – it will make your life a lot easier at the end. 





Monday, 22 July 2019

The Evaluator is a finalist!

A little while ago we mentioned that our director Kirsty Rose Parker, was nominated for an award. We are pleased to announce she has now been chosen as a finalist.

Kirsty says, "it's such an honour to be chosen as a finalist. It is an amazing feeling to be picked, and to be on a list with some other really inspirational women. Congratulations to all the other women chosen!"

The local press have also had some wonderful words to say, and Kirsty is mentioned in this article, 'Lancashire Businesswomen in Top Awards!". You can read the article here.

Kirsty has to attend a judging panel next and give a presentation and be grilled on her work. Then it's a short wait until the glittering awards ceremony at the Winter Gardens in Blackpool.

Here at The Evaluator we could not be more pleased.

Sunday, 14 July 2019

The Evaluator working with The National Festival of Making

The Evaluator is delighted to announce a new client, The National Festival of Making.

The National Festival of Making is a high quality arts and cultural festival which puts art back into manufacturing and celebrates making in all shapes and formats. It's a very busy weekend event, with 24 different sites of activity across Blackburn, Lancashire which attracts a huge range of people.

It's a big deal. Last year it won the 'Best Non Music Festival' at The UK Festival Awards.

The Evaluator is helping the team understand the artistic quality of the work, capturing the audience reaction and working on identifying the economic impact on Blackburn itself.

You can see more about the National Festival of Making here. 


Tuesday, 9 July 2019

The Evaluator working with Lancashire Wildlife Trust

The Evaluator is delighted to announce a new client, Lancashire Wildlife Trust. The Evaluator will be carrying out an independent review of a particular project 'Myplace' which is eco-therapy and uses the tag line, a natural way to wellbeing.




The project works with both children and young people and adults with mental health issues. The Evaluator is carrying out a short and fast review of the programme and evaluation data to date, in order to understand the impact of the project and help the team to write a bid for future funding.

If you would like to know more about Myplace, the link to their website is here



Friday, 21 June 2019

Evaluation Tips Series; Think about the audience right at the beginning

Here at The Evaluator we teach workshops to small community and charitable groups and show them how to plan out and carry out an evaluation of their own.


First of all, let's go right back to basics.

What is an evaluation?

A structured method to understand what happened during a project or activity. It is likely to cover who took part, what was the impact, and to answer the question: did the project or activity achieve its goal?

Sometimes it could be as simple as a conversation about what worked well and what didn’t.


One of the first things we talk about is 'Who is the audience?' We don't mean who is the audience for your activity or who is taking part, we mean - who is reading the evaluation itself, who is the audience for the evaluation?

There are four main answers...

  1. Public Document
  2. Funders
  3. Potential Funders or Sponsors
  4. Internal 


If an evaluation is designed to be a public document, then it needs to be written in a formal way. Chances are a public document will also be aimed at either current funders or potential funders.

If you are writing for  funders, they probably want to see the difference their investment has made and to know if it has achieved the aims.

If it is for potential funders or sponsors, they may be much more interested in the type of people you are working with and think about specific questions like; What they are struggling with? What is upsetting them? What do they want?  It's about finding the problem and offering a well planned solution to that problem.

If the audience is internal, you probably want to help the charity or community group understand how they can improve what they currently do, and measure what is working well and could be increased and what is not so effective and could be scaled back or changed. Sometimes an internal evaluation will be designed to help answer a specific question, for example, what can we do to recruit more participants? Or how can a specific problem be fixed. Often an internal evaluation is carried out when there is a feeling that something is not working well, or something is wrong, but there isn't a clear and obvious answer what it may be.

Overall, thinking at the beginning about what you are trying to produce and answer is a vital part of a great evaluation. Time spent planning at this stage isn't wasted.



Tuesday, 4 June 2019

Wednesday, 22 May 2019

The Evaluator working with Keighley Healthy Living

The Evaluator is delighted to announce a new partnership with the holistic health and wellbeing charity, Keighley Healthy Living.

We are going to be helping them to evaluate a new reaching communities project, Keighley CONNECTS, funded through the National Lottery Community Fund.

Keighley CONNECTS is a project which works to empower individuals through a wide range of different groups, exploring which activities make a difference to people's lives and a key part of the work we will be doing is to explore the difference between traditional tutor-led groups and those led by peer-supporters.

The Evaluator has designed a standard procedure evaluation to run across all groups; easy to complete for participants and with a unique and quick staff assessment we hope to be sharing some eye catching information soon.

You can read more about the charity, Keighley Healthy Living, here