SoGive's Gold Standard Benchmarks

SoGive’s Gold Standard Benchmarks are:

1. £5,000 per life saved
2. £50 to double someone’s consumption (spending) for one year
3. £200 to avert one year of severe depression
4. £5 to avert the suffering of one chicken who is living in very poor conditions

This means that if our models indicate that a charity is expected to be able to (e.g.) save a life for £5,000 or less, then it will have achieved the Gold Standard. (and similarly for the other benchmarks).

The rest of the document sets out the motivation for our benchmarks and how we came to our conclusions.

Motivation: how moral weights work supports SoGive’s charity ratings

SoGive performs charity analysis on a broad range of charities. Our output is an easy-to-understand rating (e.g. “Gold” or “Silver”). We aim to cover a sufficiently large number of charities that if you heard the name of a charity you could consult SoGive and have a good chance that SoGive has a write-up.

Historically we simply compared all charities to the Against Malaria Foundation (AMF). We believed then (and believe now) that this is a legitimate, philosophically well-founded comparison. We believe this because of “opportunity cost” thinking: any decision to donate to charity X is also (implicitly) a decision *not* to donate to AMF (as well as being a decision not to spend it on anything else). In principle it was never a SoGive policy to only compare charities against AMF, however our actual practice ended up being very similar to this. We chose AMF because of the robust evidence supporting its effectiveness and because of its cost-effectiveness.

However this approach suffered from a number of issues.

• Lots of people (especially those not familiar with the opportunity cost thinking set out above) felt that it was invalid to compare (e.g.) a charity helping homeless people in Coventry with AMF’s malaria net distributions
• This is problematic because SoGive’s key insight was being lost -- the whole point is that SoGive provides the data to let donors move on from “holidays for veterans vs malaria nets” to “ONE holiday for a veteran vs TWO lives saved” or “THREE homeless people helped vs SEVENTY years of depression averted”.
• While this quantification adds valuable extra information, this insight was being lost for those who thought that comparing everything to AMF was us being overly simplistic

• Lots of subtle moral assumptions were being kept implicit, and making those explicit is valuable
• Frequently the differences in cost-effectiveness were so substantial that almost any reasonable set of moral values would reach the same conclusion. However this was not always the case. For example, reasonable people have wildly differing opinions about the moral value that animals have.
• The persistent references to AMF were problematic
• If we changed our mind about AMF being a charity which meets the standard to be rated Gold, we would have a lot of work to update our analysis
• It made us look like a shill for AMF

Our approach to determining the relative weights

• Qualitative interviews with major donors known to SoGive or SoGive’s founder
• Purpose was to test question framing for understanding, as well as to identify potentially missing questions and hear some reasoning
• Recorded quantitative answers to the questions as well as notes on the above.
• Pilot general population surveys
• Tested for inconsistency between different framings
• Tested for comprehension by asking people to provide reasoning
• Noticed that people had a tendency to put 0 when (based on their reasoning) they meant “infinity”, adapted survey accordingly.
• Noticed scope insensitivity with different framings, adapted survey accordingly.

• Performed the survey on a nationally representative sample of 500 UK residents
• Requested that members of the Effective Altruism community participate in the survey; we sought input from this community because the community is characterised by people who are interested in these questions and likely to have already given them some thought. We received 13 responses
• Internal estimates from the SoGive analytical team
• SoGive team members were encouraged to come up with weights and reasoning without consulting the survey results or each other.
• They then anonymously recorded both their reasoning and weights.
• All of the SoGive team were then provided with:
• Notes and weights from the HNW interviews
• Summarised survey results from the large sample
• Survey results from the EA community
• The weights and reasoning of SoGive team
• The SoGive analysts then updated their weights based on the new evidence. They remained blind to the identity of the SoGive analysts behind each argument, but knew which proposed answers came from survey responses (as opposed to coming from SoGive analysts)
• We took the median of these updated weights.

Results of the moral weights process

Question	Median
Years of doubled consumption per life	110
Years of averted depression per life	30
Years of chicken suffering averted /life	750

Calibration

The above results give relative answers: we know how much the various answers compare with each other. 

However to get the final answer, we need an absolute amount in pounds sterling.

This is equivalent to selecting a cost per life saved, and then the rest can be calculated.

Our criteria for this were that we want to 

1. have a fairly low cost (i.e. a fairly demanding threshold); and
2. have a cost which is high enough that we do have *some* charities which meet that threshold 

We were also aiming for there to be some gap between the most cost-effective charities and the Gold Standard benchmarks, so that our estimates of them could move without us risking running out of charities which met the Gold Standard.

This led to us choosing a threshold of £5,000 per life saved.

We then calculated the threshold cost for each benchmark, and then rounded them to avoid the impression of spurious accuracy. This led to our final answers:

1. £5,000 per life saved
2. £50 to double someone’s consumption (spending) for one year
3. £200 to avert one year of severe depression
4. £5 to avert the suffering of one chicken who is living in very poor conditions

We expect to continue to review and update our moral weights and Gold Standard benchmarks.