To find the right giving opportunity we need to know how to distinguish highly effective charities from average charities. A very simple heuristic is that the best charities work on effective interventions in high priority cause areas. But this alone isn’t enough to tell us whether a charity is having the biggest impact possible. Charity evaluators have conducted detailed research and identified several key criteria according to which they recommend charities. We’ll explain the four most important criteria below.
Strength of evidence
In charity evaluation, evidence refers to empirical proof that an intervention actually has positive effects on the target group. GiveWell requires charities implement programs “that have been studied rigorously and ideally repeatedly, and whose benefits we can reasonable expect to generalize to large populations…” Evidence is important because it is the cornerstone of impact. If we can’t verify that a charity actually changes anything for the better, then donating to it is a wasted opportunity. For example, let’s consider our earlier case study on PlayPumps. If PlayPumps had been trialled before being implemented, it would have become immediately clear that the mechanism was fundamentally flawed. Millions of dollars wouldn’t have been wasted on a program with no positive impact, and numerous communities would not now be left with a more burdensome way to source fresh water.
Sometimes we can’t always demonstrate rigorously tested evidence for programs, either because the programs are difficult to test or because they are relatively new. For example, it is very difficult to find data on the success of many animal welfare programs because they haven’t been studied for their impact. In instances like these, Animal Charity Evaluators looks for interventions that are likely to be highly impactful and charities that have a strong track-record of successful campaigns.
Cost-effectiveness
Cost-effectiveness refers to “saving or improving lives as much as possible for as little money as possible.” Cost-effectiveness metrics might include “cost per life saved” or as we saw in Part Two, “cost per DALY”. Ideally, cost-effectiveness should be the most relevant criterion in evaluating charities. However, there is often insufficiently robust data available to accurately calculate cost-effectiveness. This is a product of both the complexity of the problems being solved, and the difficulty in acquiring evidence. Because of this, charity evaluators often make approximations by assigning quantitative values to benefits. As these approximations are to some extent based on value judgements and involve empirical uncertainty, charity evaluators recommend that cost-effectiveness should only play a limited role in charity evaluation.
Room for more funding
This criterion looks at whether the recommended charity requires funding beyond what it could raise for itself to continue to scale up its programs. Importantly, additional funding must increase the total impact of the charity’s work. For example, to satisfy this metric GiveWell asks “[w]hat will additional funds — beyond what a charity would raise without our recommendation — enable, and what is the value of these activities?” Animal Charity Evaluators adds “we need to make sure that even if we find something effective, that the intervention in question is bottlenecked specifically by money and that more money is what will make the intervention happen.” It’s important to understand whether additional money would be usefully spent by recommended charities to ensure our donations have an impact.
Transparency
The last key metric is how transparent potentially recommended charities are with their work. This means being open to publishing successes and failures, demonstrating that they have learnt from past mistakes, and clearly articulating their progress. GiveWell requires their recommended charities be “open to our intensive investigation process – and public discussion of their track record and progress, both the good and the bad…” The transparency of the organisation is important because without it we can’t trust that an organisation is doing the work it claims to do. Transparency indicates a charity is confident it is doing highly impactful work and flexible enough to make necessary changes to increase its impact.
Conclusion
Evaluating charities isn’t a perfect science. We’ve covered four key metrics used by charity evaluators to assess the effectiveness of charities. With limited information charity evaluators have to make judgement calls. Therefore, there are also many other considerations we could take into account to understand to which charities we should donate. For example, we could consider management structures and leadership as Animal Charity Evaluators does, or we could explore hit-based giving as the Open Philanthropy Project does.