Helen Keller International - Vitamin A Supplementation Programme Review
Executive Summary
Helen Keller International (HKI) provides technical assistance, engages in advocacy, and contributes funding to government-run vitamin A supplementation (VAS) programmes in sub-Saharan Africa.
HKI's VAS programmes are recommended by highly respected charity analysis organisation GiveWell on the basis of a Cochrane review meta-analysis, which finds that VAS reduces child mortality by 12-24%.
This document evaluates the cost effectiveness of Helen Keller International's delivery and implementation of VAS programs, while our intervention report on VAS looks at the evidence for VAS more generally. Our analysis of HKI and their VAS programmes builds on the work done by GiveWell on HKI and VAS.
This document builds on GiveWell's work, highlights some key points of contention related to the cost-effectiveness of VAS, and provides an updated cost-effectiveness estimate after taking into account these points.
In particular, across this document and our VAS intervention report, we consider:
Is there strong evidence that VAS reduces child mortality?
Should we be concerned that a large, recent VAS trial in India, DEVTA, did not have a statistically significant effect on child mortality?
Are there any substantial negative or offsetting effects of VAS?
Are HKI's VAS programmes targeted at areas where they are likely to be effective?
There is strong evidence from a Cochrane review meta-analysis that VAS reduces child mortality. There is some uncertainty around whether to use a random-effects model, which suggests that VAS reduces child mortality by 24%, or a fixed-effect model, which suggests a 12% reduction in child mortality. SoGive has reviewed an unpublished work that argues in favour of the random-effects model because of heterogeneity between studies. We therefore use the random-effects model in our cost-effectiveness model, with a small downward adjustment for internal validity.
The results from a large, recent VAS trial, DEVTA, do not show a statistically significant effect of VAS on child mortality. Although there is no clear explanation for why this result is less favourable than the other Cochrane Review RCTs, we remain confident that VAS reduces child mortality on average. We conclude that there may have been factors related to the delivery of VAS or related to the populations involved that caused differing effectiveness between DEVTA and other Cochrane review RCTs.
There are no major negative or offsetting effects of VAS.
We expect VAS programmes to be cost-effective in the regions currently targeted by HKI. Although vitamin A deficiency (VAD) prevalence is lower in current contexts compared to the Cochrane review RCTs, we believe that it remains high enough to lead to substantial reductions in child mortality as a result of VAS. In our cost-effectiveness model, we apply an external validity adjustment to the results from the Cochrane review. This downward adjustment accounts for the expected effectiveness of VAS in current HKI programmes relative to the Cochrane review RCTs, partly based on reductions in VAD prevalence, and partly based on reductions in baseline child mortality rates for VAS-preventable diseases (primarily measles and diarrhoea).
Overall, we estimate that HKI's current VAS programmes avert the death of a child for £3,377. This strong cost-effectiveness reflects the fact that supplements only cost £1.65 per child per year for two doses, in combination with the strong evidence that VAS reduces child mortality. In addition, the cost per child's life saved of £3,377 does not take into account the potential development benefits of VAS, which may lead to increases in future income as adults.
This is similar to GiveWell's estimate of a cost per life saved of £2,096. The slight difference in cost-effectiveness stems from our slightly stronger external validity downward adjustment. This downward adjustment is based on how effective we expect HKI's VAS programmes to be in current contexts relative to the Cochrane review RCTs.
In conclusion, we find that HKI's VAS programmes are more cost-effective than our Gold benchmark of saving a life for £5,000. GiveWell's analysis also suggests that HKI could use more funding than it currently expects to receive for VAS mass campaigns, up to an additional $12.7 million.
HKI's VAS programmes achieve a firm Gold rating under the SoGive methodology, with a cost per life saved that outperforms our Gold benchmark.
Some uncertainties remain, and future research may focus on:
Better understanding the heterogeneity between studies and how this affects reductions in child mortality across different trials, such as DEVTA. If these factors are likely to persist in the future, this could impact our judgement of the cost-effectiveness of VAS. In addition, this could help drive HKI's future work, improving their future cost-effectiveness.
Further reviewing estimates of vitamin A deficiency (VAD) prevalence by the Global Burden of Disease project in regions where HKI operates.
Contents
Introduction
Evidence on Vitamin A Supplementation (VAS)
Evidence on Helen Keller International's VAS Programmes
a) Are programmes targeted at areas where they are likely to be effective?
How prevalent is vitamin A deficiency in areas where HKI works?
How high are child mortality rates in areas where HKI works?
External validity adjustment
b) Are vitamin A supplements delivered to and ingested by recipients?
c) How does HKI's support affect programme outcomes?
Cost-effectiveness Analysis
Room for More Funding
Should VAS be Funded by Philanthropy or a Different Form of Funding?
Conclusion
Introduction
Helen Keller International (HKI) provides technical assistance, engages in advocacy, and contributes funding to government-run vitamin A supplementation (VAS) programmes in sub-Saharan Africa. HKI aims to reduce malnutrition, avert blindness and poor vision, and reduce child mortality.
As noted by charity analysis organisation GiveWell, HKI's support to government-run VAS programmes includes:
Technical assistance: HKI assists governments with monitoring and evaluation, training health workers and managers, policy design, planning and budgeting, and social mobilisation for VAS programmes. HKI mainly focuses on providing technical assistance at the sub-national level, particularly to districts or regions that may have low VAS coverage rates.
Advocacy: HKI encourages national governments to prioritise budgeting for and implementing VAS mass campaigns, and advocates for routine distribution of vitamin A supplements through health facilities.
Funding: HKI provides grants to governments to cover a portion of the implementation costs of VAS programmes.
In 2019, for their VAS programmes, HKI spent 26% of their expenditure on grants to governments for programme implementation (c. £1,133,000), 23% on personnel costs (c. £1,031,000), 15% on programme implementation and monitoring (c. £681,000), 3% on travel, equipment and supplies (c. £111,000), 14% on other direct costs (c. £626,000), and 19% on overhead costs (c. £836,000).
Our intervention report on VAS can be found here. Our analysis of HKI and their VAS programmes builds on the work done by GiveWell on HKI and VAS.
Evidence on Vitamin A Supplementation (VAS)
VAS is known to be an effective intervention for reducing child mortality. A Cochrane review, which is a meta-analysis of 19 randomised controlled trials (RCTs), finds that VAS reduces child mortality by 24% (95% confidence interval: 17% to 31% reduction), using a random-effects model. See our VAS intervention report for details.
Evidence on Helen Keller International's VAS Programmes
In assessing the evidence behind HKI's VAS programmes, we consider how they allocate their projects, the outcomes of those projects, and counterfactuals, i.e. what would happen if HKI did not provide VAS? We review three key questions:
Are programmes targeted at areas where they are likely to be effective?
Are vitamin A supplements delivered to and ingested by recipients?
How does HKI's support affect VAS programme outcomes?
a) Are programmes targeted at areas where they are likely to be effective?
Here, we consider two elements, which form the basis of two key parameters in our cost-effectiveness model, the "external validity adjustment" and "baseline child mortality".
How prevalent is vitamin A deficiency in areas where HKI works?
How high are child mortality rates in areas where HKI works?
How prevalent is vitamin A deficiency in areas where HKI works?
There is limited recent data on the prevalence of vitamin A deficiency (VAD) among populations targeted by HKI's VAS programmes. GiveWell's estimate, based on a model of VAD prevalence from the Global Burden of Disease project, is that the prevalence of VAD among preschool-aged children in countries where HKI works or plans to work is between 25% and 36% ("external validity" sheet, row 5). This is significantly lower than GiveWell's estimate of 59% for populations represented in the Cochrane review RCTs, on which we form our estimate for the effect of VAS on child mortality. It is therefore likely that HKI's current VAS programmes have a smaller impact on child mortality rates than the populations studied in the Cochrane review. Our cost-effectiveness model accounts for this by including a downward adjustment for "external validity".
It is possible that the less favourable results from the DEVTA trial, a recent, large RCT, are explained by lower VAD prevalence, as explained in our VAS intervention report (see "DEVTA trial"). However, it appears unlikely that VAD prevalence in a given population is a perfect predictor of VAS effectiveness (see "Summary of evidence" in our intervention report).
The prevalence of VAD in a population can be estimated using representative surveys of serum retinol concentrations or retinol-binding protein (measured in blood samples), clinically assessed eye signs of VAD (e.g., Bitot's spots, xerophthalmia), or other measures. WHO defines VAD as being indicated by a serum retinol concentration lower than 0.70 μmol/L, and severe VAD as a serum retinol concentration lower than 0.35 μmol/L.
GiveWell concludes that the prevalence of VAD among preschool-aged children remains uncertain in areas where HKI works, after considering the following factors:
Vitamin A deficiency surveys: GiveWell has listed the most recent serum retinol and retinol-binding protein surveys of preschool-aged children in countries where HKI supports or plans to support VAS mass campaign programmes.
GiveWell notes the following:
Of the nine countries where HKI supports or plans to support VAS mass campaign programmes, only two have completed nationally-representative surveys of VAD (using serum retinol or retinol-binding protein biomarkers) among preschool-aged children in the past ten years. The remaining countries completed VAD surveys more than ten years ago, or have not ever completed any.
The majority of these surveys find VAD prevalence among preschool-aged children in the "severe" range as defined by WHO (above 20% prevalence). But one of the more recent surveys, from Kenya in 2012, found prevalence rates in the "mild" range.
VAS appears to have only a temporary impact on measures of serum retinol and retinol-binding protein concentrations. Accordingly, measures of serum retinol or retinol-binding protein may not be useful for evaluating the impact of a VAS programme-instead, they may reflect whether or not dietary vitamin A intake is adequate.
Stevens et al. 2015 incorporates the most recent available VAD surveys and other relevant information (e.g. availability of animal-source foods) into a mathematical model to estimate rates of VAD as of 2013. The paper concludes that VAD was likely to be high (above 40%) in 2013 throughout sub-Saharan Africa. However, three recent VAD surveys from Sierra Leone, Malawi, and Kenya found considerably lower rates among preschool-aged children than the lower bound estimate (25%) for sub-Saharan African countries in Stevens et al. 2015 (see results for Kenya (2012, 9%), Sierra Leone (2013, 17%) and Malawi (2015-16, 4%)).
Vitamin A food fortification: Excluding the Democratic Republic of the Congo and Mali, all countries in which HKI supports or plans to support VAS mass campaign programmes mandate that vegetable oil be fortified with vitamin A. A few others mandate or allow fortification of wheat flour or sugar with vitamin A as well (see sheet "VAD where HKI works (or plans to work)"). GiveWell is unsure as to whether these food fortification programmes have affected rates of VAD among preschool-aged children in these countries. GiveWell's key findings are:
Only a handful of countries in which HKI has recently supported or plans to support VAS mass campaign programmes have implemented household- or market-level surveys testing on whether food samples are adequately fortified.
A market-level survey of vegetable oil in the city of Abidjan, Côte d'Ivoire, found that nearly all samples were adequately fortified, but other surveys we have seen found relatively low rates of adequately-fortified oil (see sheet "VAD where HKI works (or plans to work)", cells K4, K5, and K11).
Engle-Stone et al. 2017 finds that rates of VAD among preschool-aged children in two cities in Cameroon did not significantly decline between 2009 and 2012, despite vitamin A fortification of vegetable oil becoming mandatory in 2011.
Conversations with HKI and experts on vitamin A deficiency (VAD): GiveWell have discussed concerns about the lack of recent data on VAD with HKI. HKI told them that it believes it would be very surprising if VAD were no longer a problem throughout sub-Saharan Africa, especially in countries with high child mortality and malnutrition rates. Dr. Sherry Tanumihardjo, an expert on vitamin A status assessment, has said that since many vitamin A oil fortification programmes in countries in sub-Saharan Africa are relatively new, it would not be surprising if many of the programmes were not yet functioning well enough to have an impact on VAD rates among preschool-aged children. (See summary of the conversation).
GiveWell previously used the above evidence to form a subjective estimate of VAD prevalence in the countries where HKI operates, but now focuses on modelling by the Institute for Health Metrics and Evaluation (IHME):
The IHME Global Burden of Disease (GBD) project: IHME modelling calculates VAD prevalence using VAD survey results, vitamin A supplementation coverage, a socio-demographic index, and an estimate of the availability of vitamin A in a country's food supply into account (excluding fortified foods). IHME's estimates indicate that, as of 2017, VAD prevalence ranged from 25% to 36% among children aged 6- to 59-months in countries where HKI is supporting or plans to support VAS campaigns.
In accordance with GiveWell, we use IHME's modelling for our estimates of VAD prevalence. We use this to calculate the "external validity" of results from the Cochrane review meta-analysis, which forms a key parameter in our cost-effectiveness model. We expand on our external validity adjustment below.
IHME's modelling has a couple of caveats:
IHME told GiveWell that its model does not take vitamin A food fortification into account. GiveWell is uncertain how much they should expect vitamin A food fortification programmes to affect VAD prevalence.
IHME's VAD modelling takes vitamin A supplementation coverage into account, so a country with high coverage for vitamin A supplementation over time would be estimated to have lower VAD. To estimate the impact of vitamin A supplementation, we would ideally like to consider estimates of what VAD prevalence would be in the absence of supplementation programmes.
How high are child mortality rates in areas where HKI works?
Child mortality rates in countries where HKI works are lower than child mortality rates of some populations studied in the Cochrane review RCTs, but not so much lower that we would expect HKI's current programmes to be ineffective on average.
For the nine countries where HKI is supporting or plans to support VAS mass campaign programmes, GiveWell estimates a child mortality rate of 0.46% to 1.9% per year as of 2017. HKI's scope for reducing child mortality depends on the child mortality rate in the absence of their VAS programmes (baseline child mortality).
Baseline child mortality rate:
The baseline child mortality rate (row 33) relies on several assumptions, primarily:
Estimated deaths per 1,000 child years would behigher in the absence of past national VAS campaigns. The extent to which deaths per 1,000 child years would have been higher is based on the 24% reduction in mortality from the Cochrane review (see VAS intervention report for details on the Cochrane review). This means that the proportion of deaths per child (including children without VAD) would be 100% / (100%-24%) higher in the absence of VAS.
GiveWell also assumes that past national VAS campaigns were 50% as effective at reducing child mortality compared to the Cochrane review RCTs (i.e. a 24% * ½ = 12% mortality reduction). This is because the coverage rate in national campaigns is expected to have been lower than the 87% in the Cochrane review RCTs.
To verify GiveWell's best guess above, we have located estimates of national VAS coverage rates in 2017 from UNICEF's "The State of the World's Children Report". This spreadsheet summarises the estimated coverage rates in countries supported by HKI. The data in this report only refers to full coverage of VAS (two annual doses spaced 4-6 months apart), without reporting partial coverage. Consequently, this may underestimate the coverage rate of national campaigns. If we were to use these figures, instead of using GiveWell's best guess that the coverage rate was 50% as high relative to the Cochrane review RCTs, then our estimate for the baseline child mortality rate would be slightly higher on average (13.11 deaths per 1,000 child-years up from 12.37). This would make a small difference to HKI's cost per life saved, improving it by 5.7% from £2,151 (see section "Cost-effectiveness analysis") to £2,033. However, we maintain GiveWell's 50% best guess in our cost-effectiveness model given the limitation of the UNICEF report (lack of data on partial coverage of VAS).
External validity adjustment
The prevalence of VAD (the proportion of children who have VAD) in HKI's current programme regions filters into our external validity adjustment in our cost-effectiveness analysis. This is a downward adjustment based on the proportion of children whose life could potentially be saved by VAS, relative to the Cochrane review RCTs. SoGive's external validity adjustment relies on several assumptions, primarily:
Linear scaling: VAD-related mortality scales linearly as VAD prevalence changes. In other words, if VAD prevalence were to be 29.5% in current populations, compared to the estimated 59% in the Cochrane review, we would assume that VAS prevents half as many VAD-related deaths per child, all else constant.
Measles and diarrhoea deaths: VAS is likely to have an impact on mortality caused by measles and diarrhoea. The Cochrane reviewof VAS finds a statistically significant reduction in mortality due to diarrhoea, and a statistically significant reduction in measles incidence, but not measles mortality. GiveWell estimates that the effects of VAS on mortality depend mostly (85%) on its effect on measles and diarrhoea, and less so (15%) on other infectious diseases. If we change this relative weighting to anything from 75% vs 25% to 100% vs 0%, this would have little effect on our external validity adjustment.
Our external validity adjustment is calculated using the following steps:
Our ultimate goal is to find out whether expected deaths averted in a cohort changes after adjusting for external validity. This is equivalent to saying - how does the 24% mortality reduction from the Cochrane review change after adjusting for external validity?
Given a set population, the risk ratio (0.76) in the Cochrane review is defined as treatment group deaths / baseline (no treatment) deaths.
Mortality reduction = (1 - risk ratio) = 0.24 = averted deaths / baseline (no treatment) deaths
We can now estimate how much lower we expect the mortality reduction to be in areas where HKI currently works compared to 24% from the Cochrane review RCTs.
The figure below attempts to break down this formula into numbers we can calculate. We simply use total all-cause deaths to estimate baseline (no-treatment) deaths.
We assume that the term on the bottom left has stayed constant relative to the Cochrane review RCTs, because this represents the biological efficacy of VAS.
The term on the bottom right would have changed however, so we now focus on this:
NB: "Total measles & diarrhoea deaths" refers to a weighted average of deaths from measles, diarrhoea, and other infectious diseases.
Firstly, building on GiveWell's calculations, row 15 shows how much the term of the bottom right has changed compared to the Cochrane review RCTs.
However, the term on the bottom left cannot be directly calculated. To calculate this term indirectly, we use VAD prevalence data, which is available for the studies included in the Cochrane review, and can be estimated in current HKI programme regions. VAD prevalence is currently ~50% of the level found in the Cochrane review RCTs. In turn, for the numerator, we can use the reduction in VAD prevalence to approximate the reduction (since the Cochrane review) in VAD-related deaths per child (assuming our first assumption from above, linear scaling, holds).
For the denominator, if VAD-related deaths per child have fallen, measles and diarrhoea deaths per child are likely to have fallen as well, since these diseases are caused by VAD. However, we need to consider the fact that the reduction in measles and diarrhoea deaths per child may also be attributed to a reduction in other risk factors, e.g. fewer unvaccinated children, improved nutrition, and improved water and sanitation, as described here (p.e534). Step 7 explores two scenarios to quantify our external validity adjustment, depending on which risk factors predominantly drive the reduction in measles and diarrhoea deaths.
Scenario 1: Suppose that any reduction in measles and diarrhoea deaths per child since the Cochrane review is entirely attributable to a reduction in VAD prevalence. In this scenario, the reduction in the term on the bottom left would be approximately equal to the reduction in VAD-related deaths per child, which we proxy with VAD prevalence. Intuitively, VAS would be far less effective in current contexts, since there would be fewer children at risk of VAD-related deaths. The external validity adjustment in this scenario would therefore be calculated as:
x in regions where HKI operates as a proportion of x in the Cochrane review trials,
where x = VAD prevalence * Total measles and diarrhoea deaths / Total deaths
Scenario 2: Suppose that reductions in measles and diarrhoea deaths per child since the Cochrane review are equally attributable to VAD as to other causes (e.g. vaccinations, nutrition, water and sanitation). In this scenario, the ratio on the bottom left would not have changed since the Cochrane review. Intuitively, VAS would remain relatively effective in current contexts, because some of the reduction in measles and diarrhoea deaths can be explained by other causes, still leaving room for reductions in VAD-related deaths. The external validity adjustment in this scenario would therefore be calculated as:
y in regions where HKI operates as a proportion of y in the Cochrane review RCTs,
where y = Total measles and diarrhoea deaths / Total deaths
However, this scenario likely overestimates the effectiveness of VAS today. VAD-related deaths are unlikely to be entirely independent from other measles and diarrhoea deaths, because VAD may, for example, exacerbate the likelihood of death due to poor sanitation. Since VAD prevalence has almost halved since the Cochrane review, we expect that this would have helped lead to reductions in measles and diarrhoea deaths caused by poor sanitation. Therefore, for scenario 2, to take into account the impact of VAD prevalence on measles and diarrhoea deaths, we estimate the external validity adjustment for scenario 2 as the minimum of :
y in regions where HKI operates as a proportion of y in the Cochrane review RCTs
VAD prevalence in regions where HKI operates as a proportion of VAD prevalencein the Cochrane review RCTs
In other words, if VAD prevalence in a particular region where HKI operates is much lower than in the Cochrane review RCTs, we expect VAS to prevent fewer measles and diarrhoea deaths from all causes.
It is likely that the true story lies somewhere in between scenario 1 and scenario 2, since there have been some improvements (p.e534) in vaccination rates, nutrition, and water and sanitation (favouring scenario 2), but also a large reduction in VAD prevalence (favouring scenario 1). As a best guess, we take an average of the external validity adjustments from the two scenarios, which leads to an external validity adjustment of 41%, down from 43% in GiveWell's cost-effectiveness model. This makes HKI's VAS programmes look very slightly but immaterially less cost-effective.
b) Are vitamin A supplements delivered to and ingested by recipients?
For the coverage surveys from countries supported by GiveWell-directed funds in 2019, coverage ranged from 94% (Niger) to 74% (Mali), for a medium coverage rate of 85%. GiveWell puts more limited weight on coverage levels measured in the surveys reviewed before 2018 because they cover a relatively small portion of HKI's past work, and many were conducted in areas where HKI expected coverage to be low. In 2019, 75% of surveys reached a coverage rate of 80%, which is the coverage rate targeted by HKI for its VAS campaigns.
c) How does HKI's support affect programme outcomes?
To evaluate HKI's involvement in the VAS programmes, GiveWell has conducted five case studies. They found strong evidence in a few cases that HKI's funding allowed mass distributions of VAS to take place. However, these case studies may not be representative of the thirteen countries in which HKI has recently supported VAS programmes. GiveWell also did not find evidence that HKI's technical assistance enables mass distribution programmes to achieve higher coverage rates compared to a counterfactual world without HKI. This is because information on suitable comparison groups is limited, such as coverage rates in similar regions operating without HKI's support.
Cost-effectiveness Analysis
GiveWell estimates that on average the total cost to deliver a vitamin A supplement through HKI-supported mass distribution programmes is $1.09. As two supplementation rounds are needed per year, the cost per child covered per year is $2.18 (~£1.59).
Our cost-effectiveness model estimates that HKI is able to avert the death of a child for £2,151, after accounting for "excluded effects" that impact the cost per life saved (interactions between VAS and vaccines) and leverage/funging. This compares favourably to our Gold benchmark of saving a life for £5,000, even before considering the development benefits of VAS.
The model is summarised below:
Vitamin A supplementation (VAS) - Helen Keller International (HKI)CalculationsNumber of children covered with VAS with a hypothetical donationDonation to HKI (footnote 1)$100,000Cost per child covered per year (2)$2.18Total number of children covered in one year (3)114,115Meta-analysis finding on relative rate of mortality reductionReduction in all-cause mortality rates for all children treated (4)27%Mortality reduction with hypothetical donationExpected number of deaths in hypothetical cohort of children over one year in absence of VAS programme (5)1,411Internal validity adjustment (6)85%External validity adjustment (7)41%Expected deaths averted in cohort due to programme (after internal and external validity adjustments) (8)125Initial resultsInitial cost per death averted estimate (9)$1,987Downside adjustments - HKITotal downside adjustment factor (10)74%Results after downside adjustmentsExpected deaths averted in cohort due to programme (11)93Cost per death averted, after downside adjustments (12)$2,686Cost per life saved estimatesAdjustments for excluded effects that impact cost per life saved (13)2%Leverage/funging adjustment (14)-8%Cost per life saved, after downside adjustments, adjustments for excluded effects, and leverage/funging adjustments (15)$2,868 ( £2,151)
The following summary demonstrates how our internal validity adjustments impact our estimate for the cost per life saved:
|
Cost per life saved (16)$925 (£693)Internal validity adjustment (6)85%External validity adjustment (7)41%Cost per life saved after validity adjustments (15)$2,868 ( £2,151 )
Footnotes
Arbitrary donation size.
Each child receives two supplements, each of which cost $1.09.
Spending by all contributors / cost per child. Note that spending by all contributors includes other philanthropic actors and local governments in addition to HKI.
Cochrane review risk ratio (mortality reduction, 24%) / estimated coverage in the Cochrane review RCTs (87%). This tells us the expected mortality reduction rate if every delivered vitamin A supplement was also ingested by the targeted child.
This is the expected number of deaths for the cohort calculated in note 3, in the absence of VAS.
This accounts for the uncertainty about whether to use the random-effects or fixed-effect model.
This accounts for changes in VAD prevalence and changes in the composition of causes of child mortality between the regions of the Cochrane review RCTs and the regions where HKI works today.
Note 4 * note 5 * note 6 * note 7. This is the number of expected deaths averted for the cohort of children due to VAS, after internal and external validity adjustments.
Spending by all contributors / deaths averted.
This accounts for risk of wastage, quality of monitoring and evaluation, and confidence in funds being used for intended purpose.
Note 8 * note 10. This is the number of expected deaths averted after downward adjustments.
Spending by all contributors / deaths averted (after downward adjustment).
These are positive flow-on benefits from VAS, specifically that VAS can enhance the beneficial effects of vaccines.
"Leverage" is influencing others (other donors, Nutrition International, governments) to allocate their funds to VAS. "Funging" occurs if spending on VAS programmes would otherwise have come from other sources in the absence of HKI. In the case of HKI, the funging component outweighs the leverage component, and is therefore a negative effect.
Spending by all contributors / deaths averted (after adjustments for excluded effects and leverage/funging adjustments).
Cost per life saved with no internal validity adjustments
VAS may also lead to development effects in addition to reductions in mortality. Development effects refer to the fact that those who have suffered fewer health issues in their youth may receive more income in later life. SoGive's calculation of VAS development effects, adapted from GiveWell's model, is outlined here. Once the development benefits of VAS are modelled, the proportion of modelled benefits of HKI's work is 11% for development effects and 89% for mortality reduction.
Where SoGive's Cost-Effectiveness Model Differs from GiveWell's
Development effects - We slightly downgraded the development effects of VAS, explained here.
External validity - We slightly reduced the external validity adjustment. We judged that the effects of VAS in current programmes may be slightly lower relative to the Cochrane review RCTs compared to GiveWell's estimate. See section "External validity adjustment" for details.
Discount rate - We increased the discount rate for development effects from 4% to 5.6%, explained here.
Moral weights - We adjusted the moral weights between lives saved and years of doubled consumption, explained here.
Room for More Funding
GiveWell's analysis suggests HKI could use more funding than it currently expects to receive for VAS mass campaigns, up to an additional $56.9 million in 2022-24.
Should VAS be Funded by Philanthropy or a Different Form of Funding?
Philanthropic funding is likely to be the most appropriate form of funding for this intervention, as we account for leverage and funging (i.e. what would happen if VAS is not philanthropically funded by HKI) in our cost-effectiveness analysis.
Conclusion
HKI's VAS programmes achieve a firm Gold rating under the SoGive methodology, with a cost per life saved that outperforms our Gold benchmark. HKI is a transparent organisation, having shared detailed information about its programmes with GiveWell. Furthermore, HKI has room for more funding to scale up its VAS activities.
If HKI turns out to be much less cost-effective than we thought, what is the most likely reason for this?
The cost-effectiveness of HKI would be significantly reduced if the fixed-effect model is a more accurate model of the effects of VAS than the random-effects model (see SoGive's VAS intervention report for details).
The reason for the less positive results in the DEVTA study, which is the largest and one of the two most recent VAS trials, is unclear (see SoGive's VAS intervention report for details). If the reason for the less positive DEVTA results were to persist in future VAS programmes, HKI's impact may be limited. However, the Cochrane review (meta-analysis of VAS trials) includes the DEVTA trial results and still concludes that VAS substantially reduces child mortality on average.
Changes in VAD prevalence were taken into account in our external validity adjustment, estimated by the Global Burden of Disease project, but remain uncertain. If rates of VAD are lower than predicted in current climates, the impact of HKI's VAS programmes may be smaller.
Our analysis did not include consultations with biologists. Further understanding of the biological mechanisms through which VAS affects disease and mortality could potentially change our view on the effectiveness of VAS in the current landscape.
If HKI turns out to be much more cost-effective than we thought, what is the most likely reason for this?
Our model does not include the potential positive effects of VAS on night blindness and Bitot's spots. The Cochrane review of VAS trials finds a reduction in both measures, but the quality of evidence is only moderate.
The Cochrane review suggests that VAS is more effective in some cases than others. If further investigation allowed researchers and analysts to identify under which circumstances VAS is more effective, HKI could target specific areas likely to be more cost-effective.