Construct validity
Table
3 presents the results of the OLS-regression. HIV and subjective wellbeing both show significant associations with the first factor-weighted SF12 (model (1), Table
3). Using standardised beta-coefficients in column (2) we find that subjective wellbeing explains the main share in SF12 (0.389), four times the size of the effect of HIV. The health variables explain about 17% of the variation in SF12. The second factor weighted SF12 outcome measure is only significantly explained by subjective wellbeing in column (3). Subjective wellbeing has a negative sign. The beta-coefficients in column (4) show that HIV has a non-significant effect of almost zero (− 0.01) whereas subjective wellbeing has an effect of size − 0.168. Both variables explain only about 3% of the variation in the SF12.
Table 3OLS estimation: SF12 computed on the first and second factor using the 2006 sample and the pooled 2012/13 sample
Model (1) SF12 1st and 2nd Factor regressed on the Health variables using the 2006 sample |
HIV Status | −3.583*** | −0.082 | −0.424 | −0.010 |
(1.197) | | (1.217) | |
Subjective Wellbeing | 4.121*** | 0.389 | −1.780*** | −0.168 |
(0.268) | | (0.273) | |
Constant | 39.468*** | | 54.007*** | |
(2.176) | | (2.461) | |
Observations | 2069 | 2069 | 2069 | 2069 |
Covariates | YES | YES | YES | YES |
Region | YES | YES | YES | YES |
R-squared | 0.228 | 0.228 | 0.051 | 0.051 |
Model (2) SF12 1st and 2nd Factor regressed on the Health variables using the 2012/13 sample |
Normal Weight | 0.751** | 0.037 | −0.497 | −0.024 |
(0.347) | | (0.621) | |
Overweight | 0.557 | 0.019 | −0.523 | −0.017 |
(0.484) | | (0.833) | |
Obese | −0.777 | − 0.017 | −0.314 | − 0.007 |
(0.714) | | (1.313) | |
Cognitive Test Score | 0.168*** | 0.093 | 0.006 | 0.003 |
(0.032) | | (0.049) | |
Average Grip Strength | 0.117*** | 0.078 | 0.015 | 0.010 |
(0.024) | | (0.040) | |
PhQ9 Depression Scale | −0.501*** | −0.185 | −0.448*** | −0.162 |
(0.073) | | (0.108) | |
GAD7 Anxiety Scale | −1.755*** | −0.501 | − 0.587*** | − 0.163 |
(0.095) | | (0.141) | |
Subjective Wellbeing | 1.469*** | 0.150 | − 1.727*** | −0.172 |
(0.140) | | (0.253) | |
Constant | 47.889*** | | 48.550*** | |
(1.742) | | (2.817) | |
Observations | 2091 | 2091 | 2091 | 2091 |
Covariates | YES | YES | YES | YES |
Year | YES | YES | YES | YES |
Region | YES | YES | YES | YES |
R-squared | 0.675 | 0.675 | 0.112 | 0.112 |
In model (2), the BMI category “normal weight”, cognitive skills, grip strength, PhQ9, GAD7 and subjective wellbeing are significantly associated with SF12 weighted by the first factor in column (1). The associations with the outcome are positive for normal weight, cognitive skills, grip strength, and subjective wellbeing and negative for PhQ9 and GAD7. Mental health measures have the strongest association with SF12, with beta-coefficients of size − 0.185 (PhQ9), − 0.501 (GAD7) and 0.15 (subjective wellbeing) in column (2). The variables explain about 66% of the variation in SF12, with the majority of variance explained by mental health domain variables when estimating the model separately with mental health and physical health variables only (63.5% versus 19.6%).
Findings from the analysis using the SF12 with the second factor weights identify only significant negative associations of the three explanatory mental health variables. The explained variance is low (7.6%). When regressing the SF12 separately on physical and mental health explanatory variables, only about 0.2% of the variation in the SF12 is explained by the physical health domain variables and 7% is explained by the mental health domain variables.
Table
4 presents the findings from the OLS regression analysis using the SF12 mental and physical health dimensions computed on US-population weights. In model (1), HIV-status and subjective wellbeing are significantly associated with the SF12 outcomes in all columns with a positive sign for subjective wellbeing and a negative sign for HIV. Subjective wellbeing explains most variation in both the mental and physical SF12. The explained variation is higher for the physical health SF with 22.3% compared to 9.1% for mental health SF12.
Table 4OLS estimation: SF12 computed on the US-weighting for mental and physical health using the 2006 sample and the pooled 2012/13 sample
Model (1) SF12 PH and MH US-weighted regressed on the Health variables using the 2006 sample |
HIV Status | −3.108*** | −0.071 | −2.541** | − 0.057 |
(1.204) | | (1.223) | |
Subjective Wellbeing | 4.233*** | 0.401 | 2.343*** | 0.219 |
(0.272) | | (0.263) | |
Constant | 39.409*** | | 43.733*** | |
(2.233) | | (2.297) | |
Observations | 2069 | 2069 | 2069 | 2069 |
Covariates | YES | YES | YES | YES |
Region | YES | YES | YES | YES |
R-squared | 0.223 | 0.223 | 0.091 | 0.091 |
Model (2) SF12 PH and MH US-weighted regressed on the Health variables using the 2012/13 sample |
Normal Weight | 0.674 | 0.034 | 0.544 | 0.026 |
(0.452) | | (0.458) | |
Overweight | 0.112 | 0.004 | 0.884 | 0.029 |
(0.592) | | (0.645) | |
Obese | −1.342 | −0.029 | 0.170 | 0.004 |
(0.959) | | (0.988) | |
Cognitive Test Score | 0.118*** | 0.066 | 0.162*** | 0.088 |
(0.038) | | (0.038) | |
Average Grip Strength | 0.126*** | 0.084 | 0.066** | 0.043 |
(0.030) | | (0.031) | |
PhQ9 Depression Scale | −0.227*** | −0.085 | −0.624*** | −0.225 |
(0.086) | | (0.084) | |
GAD7 Anxiety Scale | −1.321*** | −0.379 | − 1.603*** | − 0.445 |
(0.113) | | (0.110) | |
Subjective Wellbeing | 2.064*** | 0.212 | 0.568*** | 0.057 |
(0.179) | | (0.186) | |
Constant | 50.200*** | | 45.184*** | |
(2.129) | | (2.155) | |
Observations | 2091 | 2091 | 2091 | 2091 |
Covariates | YES | YES | YES | YES |
Year | YES | YES | YES | YES |
Region | YES | YES | YES | YES |
R-squared | 0.495 | 0.495 | 0.485 | 0.485 |
In model (2), cognitive skills, grip strength, PhQ9, GAD7, and subjective wellbeing are significantly associated with both physical and mental health SF12. Cognitive skills, grip strength and subjective wellbeing have a positive association and PhQ9, while GAD7 has a negative association with both physical and mental SF12 measures. The GAD7 explains most of the variation with − 0.379 in physical health in column (2) and − 0.445 in mental health in column (4). The overall explained variation due to physical and mental health variables is similar for both SF12 measures: 49.5% of the physical health SF12 (column 1) and 48.5% of mental health SF12 (column 3) are explained.
Table
5 presents our findings from the Fixed Effect analysis. Columns (1) shows the results of the Malawi first factor SF12. We find that normal weight, cognitive skills and subjective wellbeing are significant and positively associated with the outcome. PhQ9 and GAD7 show a negative significant association. Mental and physical variables explain together 50% of the within individual variation, 65% of the between individual variation and 61% of the overall variation. Using separate estimation by health domain variables, 48% of the within variation, 70% of the between individual variation and 63.2% of the overall variation are explained by mental health measures. In contrast, only 4% of the within, 22.1% of the between individual and 17% of the overall variation are explained by physical health measures.
Table 5Fixed Effect estimation: (1) SF12 computed on the first and second factor and (2) SF12 computed on the US-weighting for mental and physical health using 2012/13 sample
Normal Weight | 1.278** | −1.003 | 1.748** | 0.186 |
(0.636) | (1.342) | (0.868) | (0.994) |
Overweight | 1.252 | 0.345 | 1.188 | 0.894 |
(1.055) | (1.803) | (1.345) | (1.419) |
Obese | 1.849 | 4.104 | −0.052 | 3.318 |
(1.884) | (3.239) | (2.465) | (2.540) |
Cognitive Test Score | 0.212*** | −0.036 | 0.172*** | 0.187*** |
(0.054) | (0.084) | (0.061) | (0.068) |
Average Grip Strength | 0.047 | 0.032 | 0.064 | 0.011 |
(0.045) | (0.076) | (0.052) | (0.060) |
PhQ9 Depression Scale | −0.324*** | −0.402*** | − 0.019 | −0.540*** |
(0.099) | (0.154) | (0.108) | (0.121) |
GAD7 Anxiety Scale | −1.662*** | −0.794*** | −1.238*** | −1.547*** |
(0.123) | (0.199) | (0.140) | (0.152) |
Subjective Wellbeing | 1.274*** | −1.663*** | 1.776*** | 0.529** |
(0.198) | (0.348) | (0.245) | (0.270) |
Constant | 38.882*** | −1.003 | 45.008*** | 35.619*** |
(5.338) | (1.342) | (6.630) | (6.251) |
Observations | 2091 | 2091 | 2091 | 2091 |
Individuals | 1164 | 1164 | 1164 | 1164 |
Controls | YES | YES | YES | YES |
Year | YES | YES | YES | YES |
Region | YES | YES | YES | YES |
R-squared within | 0.499 | 0.113 | 0.272 | 0.357 |
R-squared between | 0.646 | 0.062 | 0.530 | 0.395 |
R-squared overall | 0.605 | 0.068 | 0.462 | 0.378 |
Column (2) presents the results of the Malawi second factor SF12. Only the PHQ9, GAD7 and subjective wellbeing show significant and negative associations with the outcome. The overall variation explained by mental and physical health explanatory variables is 7%, within variation is 11% and between individual variation is 6%. Physical health variables alone explain only 0.04% of within individual variation, 0.02% of the between individual variation and 0% overall variation. Mental health variables explain 9.2% within individual variation, 5.8% between individual variation, and 6.7% overall variation.
Column (3) presents findings from the Fixed Effect regression with US-weighted physical health SF12. Normal weight, cognitive skills, subjective wellbeing have significant positive associations and GAD7 has a significant negative association with SF12. The health variables explain 27% of within, 53% of between individual variation and 46% in the overall variation. Compared with the physical health, the mental health measures explain more within individual physical health variation (25.7% vs. 2.9%), more between individual physical health variation (49.7 vs. 20.7%) and more overall physical health variation (42.4% vs. 15.5%).
Column (4) presents the results of the US-weighted mental health SF12. We find significant positive associations of cognitive skills and subjective wellbeing and significant negative associations of PhQ9 and GAD7 with the US-weighted mental health SF12. We find 38% of the overall variation, 36% of the within individual variation and 40% of the between individual variation explained by the health variables. Mental health variables explain more variation in the outcome than physical health variables. They explain 33.8% of the within individual (versus 2.1% for physical health variables), 53.4% of the between individual (versus 10.9% for physical health variables), and 46.6% of the overall variation (versus 7.8% for the physical health variables) in the mental health SF12.
Table A2 in the Additional file
1 presents the population norms of the SF12 measures by age-groups and gender. Mean values of the SF12 measure derived from the first factor are similar between male and females across age-groups with overlapping 95%-confidence intervals. Mean values of the instrument increase by age-groups from 49.43 in the 16–24 years age-group to 52.62 in the 55–59-years age-group. The SF12 second factor measure shows significant variation between male and females in age-groups 16–24 to 40–44 years with higher mean values for males, ranging between 51.15 (48.36) in the 40–44 age-group and 54.24 (50.39) in the 16–24 age-group for males (females). Overall, the SF12 second factor instrument decreases in age, from 51.45 in the age-group 16–24 to 46.52 in the 60+ age group.
Application to policy evaluation
We find that different SF12 mental health measures by population weights matter for the empirical analysis. Table
6 presents in model (1) quantile and average effects of the cash transfer on mental health using the Malawi-weighted SF12 mental health measure, and in model (2) findings of the analysis using the US-weighted SF12 mental health measure. Columns (1) to (5) present the findings of the quantile treatment effect analysis for each respective quantile. Column (6) presents the average treatment effect.
Table 6Quantile treatment and average effect estimation of the cash transfer on mental health using US-weights and Malawi-weights
Model (1) Estimation of quantile treatment and average effects on mental health using the Malawi-weighted SF12 mental health measure |
Treated | 4.599*** | 1.900 | 0.458 | 0.116 | 0.021 | 1.124* |
(1.690) | (1.200) | (0.852) | (0.512) | (0.296) | (0.640) |
Constant | 42.259*** | 12.347 | 35.025*** | 43.526*** | 53.298*** | 59.915*** |
(3.812) | (9.984) | (7.213) | (4.402) | (2.657) | (2.289) |
Model (2) Estimation of quantile treatment and average effects on mental health using the US-weighted SF12 mental health measure |
Treated | 5.305*** | 1.614 | 0.961 | 0.192 | −0.055 | 1.129 |
(1.696) | (1.353) | (1.163) | (0.525) | (0.082) | (0.798) |
Constant | 39.889*** | 42.957*** | 52.086*** | 56.034*** | 59.422*** | 50.014*** |
(8.008) | (6.845) | (5.170) | (2.705) | (0.790) | (3.118) |
Model (1) shows significant effects of the cash transfer programme on average of size 1.1 and for the lowest mental health quantile of size 4.6, when using the Malawi-weighted SF12 mental health measure. In contrast when using the US-weighted SF12 mental health measure in model (2), we find that the cash transfer only significantly effects the lowest quantile in mental health of size 5.3 which is 15% larger than the equivalent effect in model (1). The comparison of the findings shows that the choice of SF12 measure can have significant implications for policy analysis, with significant versus non-significant average effects dependent on the specified SF12 mental health measure. We use this evidence to advocate the choice of our validated SF12 Malawian-population weighted mental health measure for future analyses. Use of US-weights can lead to different estimates of treatment effects, on average and across quantiles.