Development doesn’t occur within a vacuum; it takes place in the real world where things are constantly shifting and changing. This means that evaluating the impact of a given project can be tricky, to say the least. The fact that it’s difficult, however, doesn’t mean we shouldn’t try to measure it. In fact, evaluation and impact assessment are key elements that we look for when considering funding decisions. It’s important to know what works, what doesn’t work and why. A good impact assessment plan will help organisations analyse and evaluate the impact of their project and how successful it’s been in bringing about change (whether anticipated or unanticipated). A convincing impact assessment is also a powerful tool that helps organisations prove the value of their work – an important consideration when looking for further funding. Occasionally, though, projects will result in failure, but evaluation and impact assessment allow us to understand why and help organisations design better and more appropriate projects for the future. Sharing the results of those failures can also help other organisations to avoid repeating the same mistakes. That’s why wherever possible, we encourage our grantees to share information with others and to be upfront about challenges or failures they encounter.
Ongoing evaluation also helps organisations change and improve their projects while they are still ongoing. If your target audience isn’t interacting with your project in the way you’d anticipated, for example, ongoing evaluation can help you identify why that’s the case and allow you to make changes to improve engagement.
In many ways, technology has made it much easier for organisations to evaluate the impact of their project. Online surveys, SMS feedback and tools such as Google Analytics offer an immediate and accurate way of accessing some basic information on the success of your project. It’s very difficult to prove causation using these tools (i.e. that your intervention is directly responsible for any change that occurs), but they do offer quick and easy ways of garnering feedback and statistics on the reach and success of your project. When combined with other techniques, they can produce insights that go beneath the surface and help organisations understand what more they can do or what other factors need to be taken into account in order to bring about the desired change.
Two of Indigo’s grantees spring to mind when looking at impact assessment. The first is the Liverpool School of Tropical Medicine, whose IntHEC project on sexual and reproductive healthcare involved a huge baseline survey, control communities and randomised cluster trials. Obviously, such rigorous, scientific methods aren’t available to the majority of Indigo’s grantees and we recognise that a £10,000 grant only allows for limited evaluation and assessment. Still, organisations that are committed to analysis and reflection can achieve much on a small budget. iCow, for example, are continually conducting surveys with their users and seeking their feedback on how to improve their services. By conducting a baseline survey with dairy farmers using the platform and then repeating the survey seven months later, the iCow team were able to see how their services were impacting upon farmers and found that 82% were still on the platform, with 42% of users reporting increased incomes since joining the platform. While iCow’s data is not of a similarly rigorous standard as LSTM’s, it nevertheless shows a commitment to measuring change and improving the service. This in itself is a good indication that an organisation is thinking about how it can improve and how it intends to demonstrate its impact.
In the penultimate blog post of this series, I’ll look at why we like our grantees to be open and transparent.