- see [[220303_135802 fact-checker analyses|initial fact-checker analyses]] ![[dv_fc_retweets_oct17-oct24_model_interact_3bins 1.png|800]] # Model comparison ```r # fit full and reduced (no condition) models (each model accounts for blocking) m101 <- lm(mean_t1 ~ conditionC * mean_t0_bin + as.factor(block), dt1[domain_type == "overall"]) m102 <- lm(mean_t1 ~ mean_t0_bin + as.factor(block), dt1[domain_type == "overall"]) # compare the two models > anova(m102, m101) Analysis of Variance Table Model 1: mean_t1 ~ mean_t0_bin + as.factor(block) Model 2: mean_t1 ~ conditionC * mean_t0_bin + as.factor(block) Res.Df RSS Df Sum of Sq F Pr(>F) 1 27462 6956483 2 27459 6953468 3 3015.2 3.9689 0.007718 ** ``` # Coefficient/effect of treatment for each bin - each row is the coefficient for condition/treatment for one bin (rows/bins 1 to 3) - contrast `t - c`: treatment minus control - accounted for blocking ```r > pairwise_contrasts contrast mean_t0_bin estimate SE df t.ratio p.value 1: t - c _1 0.84529024 0.3195983 27459 2.6448520 0.008177316 2: t - c _2 0.04856916 0.3202193 27459 0.1516747 0.879444627 3: t - c _3 -0.74376142 0.3205835 27459 -2.3200240 0.020346872 ``` # Predict whether user quality is below 50 - if quality during campaign is < 50, assign 1, else 0 ```r dt1[, user_low_quality := ifelse(mean_t1 < 50, 1, 0)] # logistic regression Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -1.1517369 0.0142815 -80.645 <2e-16 *** conditionC 0.0107100 0.0285630 0.375 0.7077 mean_t0C -0.0481450 0.0007932 -60.699 <2e-16 *** conditionC:mean_t0C 0.0030506 0.0015864 1.923 0.0545 . # interaction? see figure below # account for blocking m301 <- feglm(user_low_quality ~ conditionC * mean_t0C + as.factor(block), dt1, family = 'binomial') summary(m301) term estimate std.error statistic p.value 1: (Intercept) -1.022614358 1.26560987 -0.8080013 0.41909 2: conditionC 0.018978089 0.03627339 0.5231959 0.60084 3: mean_t0C -0.042917939 0.00110691 -38.7727432 0.00000 4: conditionC:mean_t0C 0.003855537 0.00203961 1.8903309 0.05871 # possible interaction? ``` For users who shared lower-quality stuff (red line) in pre-campaign, treatment made them *less likely* to belong to the "bad group" (quality < 50) during the campaign. But opposite effect for users who share higher-quality stuff (turquoise line)! But consistent with all our interaction results so far. ![[Pasted image 20220304092352.png]]