Have to first reduce/aggregate each subj's data to a single row (so will lack statistical power to detect treatment effects and doesn't account for item effects) - `y` (truth ratings for **false** headlines) for each subject: $\bar{y}_{repeated} - \bar{y}_{novel}$ - only 275 rows/subjects As with results from mixed-effects models, adjusting for covariates matters. ```r # causal effect after adjusting for 11 covariates > average_treatment_effect(cf) estimate std.err -0.08206909 0.08725634 # huge SE, but correct sign # explore/compare: causal effect adjusting for just age and gender > average_treatment_effect(cf) estimate std.err -0.001629907 0.084440179 # no treatment effect at all, so adjusting for covariates indeed mattered ``` # forest feature importance ```r covariate imp 1: tenuredays 0.275882666 # most predictive of treatment effect 2: gap_days 0.205919552 3: conspiracy 0.133391372 4: age 0.105974972 5: aot 0.101171554 6: covid_concern 0.060441118 7: genderMale 0.030862710 8: domainMedia & Ent 0.022562396 9: domainProfessional Services / Industry 0.012170469 10: domainSocial Media 0.010141687 11: regionSEA 0.007875551 12: countryGR 0.007493924 13: domainHi-Tech 0.006852252 14: countryPH 0.005138410 15: roleTeam Leader 0.003737543 16: roleTeammate 0.002965230 17: regionLATAM 0.002025386 18: countryTW 0.001920672 19: regionIN 0.001754025 20: countryIN 0.001718512 21: domainSupport 0.000000000 covariate imp ``` # heterogeneous treatment effects (HTE) There's some evidence for HTE. If area-under-curve(AUC) in the fig below is positive, then there's HTE. ![[s20220801_115003.png]] # linear HTE effects ```r # testing for LINEAR effects! Best linear projection of the conditional average treatment effect. Confidence intervals are cluster- and heteroskedasticity-robust (HC3): Estimate Std. Error t value Pr(>|t|) (Intercept) 1.7795e-01 7.6960e-01 0.2312 0.817317 age -2.5360e-03 1.2565e-02 -0.2018 0.840198 tenuredays -9.0839e-05 1.9319e-04 -0.4702 0.638596 aot 5.7687e-02 6.6946e-02 0.8617 0.389622 covid_concern -1.3539e-01 6.4589e-02 -2.0961 0.037011 * conspiracy -6.0817e-03 5.4675e-02 -0.1112 0.911514 gap_days 1.0507e-01 3.4180e-02 3.0741 0.002329 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` ## probe non-linear HTE with partial dependence plots Subjs who have been moderators for longer show **weaker** treatment effects. ![[1659376370.png]] The longer the no. of days between wave1 and wave2 complete, the weaker the treatment effect. ![[1659376440.png]] Higher AOT associated with weaker treatment effects. ![[1659376557.png]] Greater COVID concern associated with stronger treatment effects. ![[1659376595.png]]