- [iffy domains](https://docs.google.com/spreadsheets/d/1ck1_FZC-97uDLIlvRJDTrGqBk0FuDe9yHkluROgpGS8/edit#gid=1144285784) - analyses below are for **retweets**, Oct 17 to Oct 24 Iffy misinfome domain ratings ![[s20220310_175335.png]] # Count analyses (quasi-poisson) Model: `count_time1 ~ condition[-0.5/0.5] * count_time0` - `count_time0LC`: `log(count_time0 + 1)` ```r # OLS m <- glm(count_t1 ~ conditionC * count_t0LC, dt1[domain_type == "overall"], family = "quasipoisson") summary(m) Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.8742605 0.0273824 31.928 <2e-16 *** conditionC 0.0185836 0.0547648 0.339 0.734 count_t0LC 0.8431191 0.0164322 51.309 <2e-16 *** conditionC:count_t0LC -0.0007253 0.0328643 -0.022 0.982 # robust SEs z test of coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) 0.87426049 0.02743417 31.8676 <2e-16 *** conditionC 0.01858362 0.05486833 0.3387 0.7348 count_t0LC 0.84311912 0.01723380 48.9224 <2e-16 *** conditionC:count_t0LC -0.00072532 0.03446761 -0.0210 0.9832 # account for blocking and robust SE > mb <- feglm(count_t1 ~ conditionC * count_t0LC | block, dt1[domain_type == "overall"], family = 'quasipoisson') NOTE: 667 fixed-effects (3,757 observations) removed because of only 0 outcomes. > summary(mb, vcov = "HC1") GLM estimation, family = quasipoisson, Dep. Var.: count_t1 Observations: 29,131 Fixed-effects: block: 4,757 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.013074 0.045290 0.288674 0.77283 count_t0LC 0.351705 0.028403 12.382643 < 2.2e-16 *** conditionC:count_t0LC 0.009051 0.027902 0.324405 0.74563 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Squared Cor.: 0.414064 ``` # Continuous ratings analysis - only bad domains (hence quality upper bound is 30) - see also [[220301_145044 user ECDFs#iffy misinfome 471 bad domains|iffy misinfome user ECDFs]] Treatment condition shared somewhat higher quality bad stuff? ```r # account for blocking and robust SE > m1 <- feols(mean_t1 ~ conditionC * mean_t0C | block, dt1[domain_type == "overall"]) > summary(m1, vcov = "HC1") OLS estimation, Dep. Var.: mean_t1 Observations: 32,888 Fixed-effects: block: 5,424 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.063683 0.035724 1.782611 0.074661 . # treatment quality increased? mean_t0C 0.424388 0.007206 58.889661 < 2.2e-16 *** conditionC:mean_t0C 0.001111 0.012414 0.089486 0.928696 ``` # 3-bins analysis No interactions ```r # bin mean and size > dt1[, .(mean_t0 = mean(mean_t0), n = .N), keyby = .(bin = mean_t0_bin)] bin mean_t0 n 1: _1 3.181468 10981 2: _2 6.739604 14248 3: _3 12.928745 7659 > m3.1 <- feols(mean_t1 ~ conditionC * mean_t0_bin | block, dt1[domain_type == "overall"], vcov = "HC1") > m3.1 OLS estimation, Dep. Var.: mean_t1 Observations: 32,888 Fixed-effects: block: 5,424 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.095950 0.060811 1.577822 0.11462 mean_t0_bin_2 1.133373 0.051965 21.810400 < 2.2e-16 *** mean_t0_bin_3 3.734485 0.070092 53.279900 < 2.2e-16 *** conditionC:mean_t0_bin_2 -0.084701 0.078398 -1.080399 0.27997 conditionC:mean_t0_bin_3 0.027759 0.127958 0.216943 0.82825 # model comparisons (w/o condition) > m101 <- feols(mean_t1 ~ conditionC * mean_t0_bin | block, dt1[domain_type == "overall"]) > m102 <- feols(mean_t1 ~ mean_t0_bin | block, dt1[domain_type == "overall"]) > test_wald(m102, m101) Name | Model | df | df_diff | F | p ---------------------------------------------- m102 | fixest | 32886 | | | m101 | fixest | 32883 | 3.00 | 1.88 | 0.130 # no effect ``` # Exploratory stuff with other predictors - what happens if we also add retweet count during time0 and time1 in the models predicting mean quality at time1? ```r # predictors: condition, qualityt0, count0(log), countt1(log) > m <- feols(mean_t1 ~ conditionC * mean_t0C + t1_retweetLC + t0_retweetLC | block, dt2[domain_type == "overall"]) > summary(m, vcov = "HC1") OLS estimation, Dep. Var.: mean_t1 Observations: 32,888 Fixed-effects: block: 5,424 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.065554 0.035654 1.838597 6.5985e-02 . mean_t0C 0.425248 0.007173 59.284398 < 2.2e-16 *** t1_retweetLC 0.109740 0.013510 8.122573 4.7539e-16 *** t0_retweetLC -0.121054 0.015206 -7.961186 1.7694e-15 *** conditionC:mean_t0C 0.000786 0.012367 0.063561 9.4932e-01 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 2.93587 Adj. R2: 0.285441 Within R2: 0.19053 # interactions with condition > m <- feols(mean_t1 ~ conditionC * (mean_t0C + t1_retweetLC + t0_retweetLC) | block, dt2[domain_type == "overall"]) > summary(m, vcov = "HC1") OLS estimation, Dep. Var.: mean_t1 Observations: 32,888 Fixed-effects: block: 5,424 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.065544 0.035664 1.837814 6.6101e-02 . mean_t0C 0.425252 0.007174 59.279970 < 2.2e-16 *** t1_retweetLC 0.109689 0.013510 8.119062 4.8931e-16 *** t0_retweetLC -0.120971 0.015201 -7.958299 1.8111e-15 *** conditionC:mean_t0C 0.000779 0.012361 0.063026 9.4975e-01 conditionC:t1_retweetLC -0.005827 0.023615 -0.246769 8.0509e-01 conditionC:t0_retweetLC 0.005363 0.028757 0.186492 8.5206e-01 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 2.93586 Adj. R2: 0.285391 Within R2: 0.190533 # all interactions > m <- feols(mean_t1 ~ conditionC * mean_t0C * t0_retweetLC * t1_retweetLC | block, dt2[domain_type == "overall"]) > summary(m, vcov = "HC1") OLS estimation, Dep. Var.: mean_t1 Observations: 32,888 Fixed-effects: block: 5,424 Standard-errors: Heteroskedasticity-robust Estimate Std. Error t value Pr(>|t|) conditionC 0.053694 0.037255 1.441248 1.4953e-01 mean_t0C 0.434035 0.008971 48.381519 < 2.2e-16 *** t0_retweetLC -0.110281 0.014052 -7.848166 4.3747e-15 *** t1_retweetLC 0.072430 0.012787 5.664168 1.4922e-08 *** conditionC:mean_t0C -0.030011 0.016387 -1.831423 6.7048e-02 . conditionC:t0_retweetLC 0.003427 0.026501 0.129329 8.9710e-01 mean_t0C:t0_retweetLC -0.035603 0.007491 -4.752873 2.0156e-06 *** conditionC:t1_retweetLC 0.000317 0.022498 0.014071 9.8877e-01 mean_t0C:t1_retweetLC 0.108930 0.004474 24.347782 < 2.2e-16 *** t0_retweetLC:t1_retweetLC -0.011408 0.003744 -3.047403 2.3105e-03 ** conditionC:mean_t0C:t0_retweetLC 0.026281 0.014571 1.803740 7.1283e-02 . conditionC:mean_t0C:t1_retweetLC -0.009928 0.008801 -1.128131 2.5927e-01 conditionC:t0_retweetLC:t1_retweetLC 0.000345 0.007288 0.047332 9.6225e-01 mean_t0C:t0_retweetLC:t1_retweetLC 0.002320 0.002956 0.784844 4.3255e-01 conditionC:mean_t0C:t0_retweetLC:t1_retweetLC 0.014693 0.005760 2.550745 1.0755e-02 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 2.88318 Adj. R2: 0.310605 Within R2: 0.219322 ```