- [[220826_095953 ukraine twitter ad preliminary results|previous round results]] - biolabs dataset: 12763 tweets - biolabs + latest datasets: 151071 tweets - 5117 users had at least 1 bad tweet - very skewed distribution - userid `1369039188` has 194 bad tweets on one day (of 301 tweets shared) - 38376 bad tweets shared by 17k users over 29 days - option 2: 63157 bad tweets shared over 29 days - also included tweets that were shared that were shared when the campaigns weren't running (model only valid for **overall model**, not daily model) # results - winsorize 95% - added IO classifier score as covariate (at least one covariate required for the the two-stage least-squares bootstrapping model below to run) - outcome: count of bad tweets summary of results - POSITIVE condition effects?! - smaller SEs relative to [[220826_095953 ukraine twitter ad preliminary results|previous results]] ## intent-to-treat models ```r # overall model (only block fixed effect) > feglm(bad_tweet_count_win ~ condition + classifier_score | block, cluster = "block", data = dt_overall_active, family = "quasipoisson") NOTE: 704 fixed-effects (3,328 observations) removed because of only 0 outcomes. GLM estimation, family = quasipoisson, Dep. Var.: bad_tweet_count_win Observations: 12,545 Fixed-effects: block: 2,219 Standard-errors: Clustered (block) Estimate Std. Error t value Pr(>|t|) conditiont 0.028811 0.040577 0.710027 0.47776 classifier_score 2.203687 0.229411 9.605857 < 2.2e-16 *** # daily model (block and day fixed effects) > feglm(bad_tweet_count_win ~ condition + classifier_score | block + sub_campaign, cluster = "block", data = dt_daily_active, family = "quasipoisson") NOTE: 704/14 fixed-effects (207,915 observations) removed because of only 0 outcomes. GLM estimation, family = quasipoisson, Dep. Var.: bad_tweet_count_win Observations: 140,484 Fixed-effects: block: 2,219, sub_campaign: 13 Standard-errors: Clustered (block) Estimate Std. Error t value Pr(>|t|) conditiont 0.013458 0.04908 0.274213 7.8395e-01 classifier_score 2.157271 0.26560 8.122242 7.5121e-16 *** ``` ```r # option 2 model (ALL tweets, even when campaigns weren't running) # overall model (block fixed effect) > feglm(bad_tweet_count_win ~ condition + classifier_score | block, cluster = "block", data = dt_overall_active, family = "quasipoisson") NOTE: 524 fixed-effects (2,409 observations) removed because of only 0 outcomes. GLM estimation, family = quasipoisson, Dep. Var.: bad_tweet_count_win Observations: 13,464 Fixed-effects: block: 2,399 Standard-errors: Clustered (block) Estimate Std. Error t value Pr(>|t|) conditiont 0.042513 0.038368 1.10801 0.26797 classifier_score 2.153527 0.217764 9.88929 < 2.2e-16 *** # interaction > feglm(bad_tweet_count_win ~ condition * classifier_score | block, cluster = "block", data = dt_overall_active, family = "quasipoisson") NOTE: 524 fixed-effects (2,409 observations) removed because of only 0 outcomes. GLM estimation, family = quasipoisson, Dep. Var.: bad_tweet_count_win Observations: 13,464 Fixed-effects: block: 2,399 Standard-errors: Clustered (block) Estimate Std. Error t value Pr(>|t|) condition 0.076003 0.046527 1.63353 0.10249 classifier_score 2.159436 0.217717 9.91856 < 2.2e-16 *** condition:classifier_score -0.179505 0.139634 -1.28554 0.19873 ``` ### ITT for each adgroup - [[220914_102436 27 ukraine adgroups characteristics|see adgroup characteristics]] ![[adgroup_itt_condition2.png]] option 2 model with all tweets ![[adgroup_itt_condition2_alltweets.png]] ## two-stage least-squares with bootstrapped SEs ```r # overall model (block fixed effect) GLM estimation, family = quasipoisson, Dep. Var.: Y Observations: 12,545 Fixed-effects: block: 2,219 Standard-errors: Custom Estimate Std. Error t value Pr(>|t|) x1 2.202785 0.192008 11.472337 < 2.2e-16 *** D 0.283980 0.496630 0.571815 0.56746 # condition effect stage1_resids -1.101316 1.266146 -0.869817 0.38442 # daily model (blocked + day fixed effects) GLM estimation, family = quasipoisson, Dep. Var.: Y Observations: 140,484 Fixed-effects: block: 2,219, sub_campaign: 13 Standard-errors: Custom Estimate Std. Error t value Pr(>|t|) x1 2.156790 0.253095 8.521675 < 2.2e-16 *** D 0.053241 0.390899 0.136200 0.89166 stage1_resids -0.824205 0.757285 -1.088367 0.27643 ``` ```r # option 2 model with all tweets (block fixed effect) GLM estimation, family = quasipoisson, Dep. Var.: Y Observations: 13,464 Fixed-effects: block: 2,399 Standard-errors: Custom Estimate Std. Error t value Pr(>|t|) x1 2.152916 0.228394 9.426313 < 2.2e-16 *** D 0.420837 0.444224 0.947353 0.34348 # condition effect stage1_resids -1.416404 1.403453 -1.009228 0.31289 # interaction model GLM estimation, family = quasipoisson, Dep. Var.: Y Observations: 13,464 Fixed-effects: block: 2,399 Standard-errors: Custom Estimate Std. Error t value Pr(>|t|) x1 2.209811 0.258622 8.544552 < 2.2e-16 *** D 0.628309 0.549351 1.143728 0.25276 stage1_resids -1.367799 1.176519 -1.162581 0.24502 x1:D -1.072977 1.368777 -0.783895 0.43312 ```