230220_230732 study 2 results

- [prereg](https://docs.google.com/document/d/1NnlXghBVoA7bGCedyKJBY30J3BB4_cXHZkhZB1axQ2c/edit) ```r n = 199 # completed study only once # veracityC: -0.5 (false), 0.5 (true) # repetitionC: -0.5 (new), 0.5 (repeated) # covariates are z-scored ``` # bayesian models ## accuracy rating coded 1-6 - linear regression ```r # basic model Estimate Est.Error l-95% CI u-95% CI Intercept 3.32 0.09 3.14 3.48 veracityC 0.66 0.10 0.46 0.87 repetitionC 0.10 0.03 0.04 0.15 veracityC:repetitionC -0.03 0.05 -0.13 0.07 # plus covariates Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS Intercept 3.33 0.08 3.17 3.49 1.00 2553 5509 veracityC 0.66 0.10 0.45 0.86 1.00 4492 9430 repetitionC 0.10 0.03 0.04 0.15 1.00 31076 27668 ageZ -0.06 0.07 -0.19 0.08 1.00 2913 6130 genderZ 0.14 0.07 0.00 0.28 1.00 2374 5449 aotZ -0.47 0.07 -0.62 -0.33 1.00 2776 5683 covidconcernZ -0.12 0.08 -0.27 0.03 1.00 2856 6036 sesZ 0.16 0.07 0.02 0.30 1.00 2918 5588 attentionZ -0.11 0.07 -0.26 0.03 1.00 2623 5381 educationZ -0.03 0.07 -0.17 0.11 1.00 3041 5963 godZ 0.15 0.07 0.01 0.30 1.00 3117 6201 veracityC:repetitionC -0.02 0.05 -0.13 0.08 1.00 32172 29378 veracityC:ageZ 0.01 0.06 -0.11 0.12 1.00 6289 12848 veracityC:genderZ -0.10 0.06 -0.21 0.02 1.00 6765 13745 veracityC:aotZ 0.33 0.06 0.20 0.45 1.00 7145 13612 veracityC:covidconcernZ 0.18 0.06 0.06 0.31 1.00 7010 14283 veracityC:sesZ -0.05 0.06 -0.16 0.07 1.00 6754 13116 veracityC:attentionZ 0.08 0.06 -0.04 0.20 1.00 7144 13332 veracityC:educationZ 0.09 0.06 -0.03 0.21 1.00 7304 12813 veracityC:godZ -0.03 0.06 -0.15 0.09 1.00 6890 14087 # also control for repetition * covariates Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Intercept 3.33 0.08 3.17 3.49 1.00 2405 veracityC 0.66 0.10 0.45 0.85 1.00 3839 repetitionC 0.10 0.03 0.04 0.15 1.00 26029 ageZ -0.06 0.07 -0.19 0.08 1.00 2419 genderZ 0.14 0.07 0.01 0.28 1.00 2326 aotZ -0.47 0.07 -0.61 -0.32 1.00 2335 covidconcernZ -0.11 0.08 -0.25 0.04 1.00 2863 sesZ 0.15 0.07 0.01 0.29 1.00 2203 attentionZ -0.11 0.07 -0.26 0.03 1.00 2220 educationZ -0.04 0.07 -0.18 0.11 1.00 2283 godZ 0.16 0.07 0.01 0.30 1.00 2742 veracityC:repetitionC -0.02 0.05 -0.13 0.08 1.00 27171 veracityC:ageZ 0.01 0.06 -0.10 0.13 1.00 6383 veracityC:genderZ -0.10 0.06 -0.21 0.02 1.00 5750 veracityC:aotZ 0.33 0.06 0.21 0.45 1.00 5392 # veracityC:covidconcernZ 0.19 0.06 0.06 0.31 1.00 5228 # veracityC:sesZ -0.05 0.06 -0.17 0.06 1.00 5655 veracityC:attentionZ 0.08 0.06 -0.04 0.20 1.00 5851 veracityC:educationZ 0.09 0.06 -0.02 0.21 1.00 5731 veracityC:godZ -0.03 0.06 -0.15 0.10 1.00 5547 repetitionC:ageZ 0.02 0.03 -0.04 0.07 1.00 25341 repetitionC:genderZ 0.02 0.03 -0.04 0.07 1.00 24965 repetitionC:aotZ 0.03 0.03 -0.03 0.09 1.00 26227 repetitionC:covidconcernZ 0.03 0.03 -0.03 0.09 1.00 20799 repetitionC:sesZ -0.05 0.03 -0.10 0.01 1.00 25063 repetitionC:attentionZ 0.00 0.03 -0.06 0.06 1.00 24150 repetitionC:educationZ 0.01 0.03 -0.05 0.06 1.00 24759 repetitionC:godZ 0.00 0.03 -0.05 0.06 1.00 26686 # add hire (duration since hire) covariate Estimate Est.Error Q2.5 Q97.5 Intercept 3.3320 0.0821 3.1734 3.4928 veracityC 0.6578 0.1015 0.4593 0.8578 repetitionC 0.0957 0.0276 0.0411 0.1496 ageZ -0.0731 0.0709 -0.2101 0.0659 genderZ 0.1363 0.0700 -0.0007 0.2733 aotZ -0.4639 0.0735 -0.6082 -0.3192 covidconcernZ -0.1159 0.0774 -0.2672 0.0359 sesZ 0.1561 0.0703 0.0182 0.2949 attentionZ -0.1268 0.0755 -0.2752 0.0234 educationZ -0.0363 0.0710 -0.1777 0.1021 godZ 0.1528 0.0739 0.0057 0.2980 hireZ 0.0802 0.0687 -0.0508 0.2187 # nothing veracityC:repetitionC -0.0202 0.0533 -0.1251 0.0838 veracityC:ageZ 0.0254 0.0591 -0.0916 0.1413 veracityC:genderZ -0.0903 0.0585 -0.2038 0.0258 veracityC:aotZ 0.3236 0.0609 0.2035 0.4430 veracityC:covidconcernZ 0.1902 0.0648 0.0637 0.3173 veracityC:sesZ -0.0589 0.0592 -0.1734 0.0590 veracityC:attentionZ 0.0961 0.0630 -0.0273 0.2199 veracityC:educationZ 0.0974 0.0600 -0.0214 0.2144 veracityC:godZ -0.0246 0.0619 -0.1455 0.0959 veracityC:hireZ -0.0882 0.0580 -0.2012 0.0260 # nothing repetitionC:ageZ 0.0175 0.0282 -0.0386 0.0727 repetitionC:genderZ 0.0154 0.0276 -0.0381 0.0703 repetitionC:aotZ 0.0298 0.0292 -0.0275 0.0868 repetitionC:covidconcernZ 0.0277 0.0302 -0.0315 0.0876 repetitionC:sesZ -0.0460 0.0275 -0.0997 0.0079 repetitionC:attentionZ 0.0020 0.0294 -0.0558 0.0601 repetitionC:educationZ 0.0058 0.0282 -0.0492 0.0614 repetitionC:godZ 0.0045 0.0289 -0.0522 0.0617 repetitionC:hireZ -0.0025 0.0276 -0.0568 0.0517 # nothing ``` ## accuracy rating coded [0,1] (0: 1-3, 1: 4-6) - logistic regression ```r # base model Estimate Est.Error l-95% CI u-95% CI Intercept -0.14 0.24 -0.61 0.32 veracityC 1.36 0.24 0.89 1.84 # exp(beta) = 3.896193 repetitionC 0.28 0.08 0.13 0.44 # exp(beta) = 1.32313 veracityC:repetitionC -0.18 0.15 -0.48 0.12 # plus covariates Estimate Est.Error Q2.5 Q97.5 Intercept -0.1097 0.2145 -0.5249 0.3125 veracityC 1.3619 0.2386 0.8874 1.8321 repetitionC 0.2822 0.0792 0.1288 0.4389 ageZ -0.1667 0.1909 -0.5463 0.2082 genderZ 0.3361 0.1955 -0.0469 0.7193 aotZ -1.1288 0.2071 -1.5396 -0.7320 covidconcernZ -0.2159 0.2118 -0.6351 0.2003 sesZ 0.3836 0.1962 0.0014 0.7744 attentionZ -0.3968 0.2081 -0.8087 0.0100 educationZ -0.0365 0.2005 -0.4348 0.3521 godZ 0.4231 0.2034 0.0215 0.8160 veracityC:repetitionC -0.1969 0.1514 -0.4929 0.1016 veracityC:ageZ 0.1780 0.1308 -0.0745 0.4343 veracityC:genderZ -0.2610 0.1355 -0.5280 0.0053 veracityC:aotZ 0.9200 0.1544 0.6264 1.2322 veracityC:covidconcernZ 0.4200 0.1434 0.1448 0.7077 veracityC:sesZ -0.1359 0.1358 -0.4070 0.1319 veracityC:attentionZ 0.0613 0.1429 -0.2163 0.3497 veracityC:educationZ 0.0501 0.1430 -0.2316 0.3256 veracityC:godZ -0.3061 0.1408 -0.5855 -0.0324 # also control for repetition * covariates Estimate Est.Error Q2.5 Q97.5 Intercept -0.1071 0.2139 -0.5288 0.3112 veracityC 1.3622 0.2350 0.9070 1.8239 repetitionC 0.2957 0.0823 0.1337 0.4570 ageZ -0.1457 0.1959 -0.5372 0.2351 genderZ 0.3563 0.1998 -0.0383 0.7484 aotZ -1.1718 0.2103 -1.5889 -0.7624 covidconcernZ -0.2080 0.2185 -0.6378 0.2196 sesZ 0.3525 0.1991 -0.0367 0.7428 attentionZ -0.3934 0.2119 -0.8129 0.0137 educationZ -0.0684 0.2019 -0.4643 0.3267 godZ 0.4620 0.2106 0.0538 0.8830 veracityC:repetitionC -0.1666 0.1537 -0.4668 0.1363 veracityC:ageZ 0.1751 0.1321 -0.0810 0.4365 veracityC:genderZ -0.2677 0.1371 -0.5402 -0.0018 veracityC:aotZ 0.9243 0.1561 0.6265 1.2414 veracityC:covidconcernZ 0.4147 0.1458 0.1330 0.7067 veracityC:sesZ -0.1327 0.1358 -0.4018 0.1307 veracityC:attentionZ 0.0641 0.1440 -0.2169 0.3506 veracityC:educationZ 0.0533 0.1433 -0.2302 0.3325 veracityC:godZ -0.3114 0.1428 -0.5960 -0.0378 repetitionC:ageZ 0.0427 0.0777 -0.1114 0.1953 repetitionC:genderZ 0.0439 0.0826 -0.1173 0.2071 repetitionC:aotZ -0.1122 0.0944 -0.3024 0.0690 repetitionC:covidconcernZ 0.0073 0.0842 -0.1574 0.1713 repetitionC:sesZ -0.1003 0.0817 -0.2580 0.0617 repetitionC:attentionZ 0.0100 0.0872 -0.1621 0.1802 repetitionC:educationZ -0.1045 0.0870 -0.2735 0.0677 repetitionC:godZ 0.0904 0.0835 -0.0720 0.2544 # add hire (duration since hire) covariate Estimate Est.Error Q2.5 Q97.5 Intercept -0.1108 0.2147 -0.5328 0.3086 veracityC 1.3711 0.2402 0.9027 1.8502 repetitionC 0.2932 0.0837 0.1304 0.4582 ageZ -0.1827 0.1992 -0.5778 0.2050 genderZ 0.3433 0.2019 -0.0575 0.7358 aotZ -1.1583 0.2090 -1.5755 -0.7533 covidconcernZ -0.2126 0.2165 -0.6422 0.2128 sesZ 0.3565 0.2023 -0.0427 0.7477 attentionZ -0.4340 0.2148 -0.8633 -0.0203 educationZ -0.0840 0.2048 -0.4910 0.3164 godZ 0.4635 0.2124 0.0450 0.8836 hireZ 0.2034 0.1979 -0.1804 0.5949 # nothing veracityC:repetitionC -0.1664 0.1555 -0.4735 0.1399 veracityC:ageZ 0.2059 0.1347 -0.0557 0.4741 veracityC:genderZ -0.2576 0.1384 -0.5312 0.0145 veracityC:aotZ 0.9145 0.1579 0.6137 1.2325 veracityC:covidconcernZ 0.4173 0.1450 0.1390 0.7067 veracityC:sesZ -0.1343 0.1384 -0.4090 0.1408 veracityC:attentionZ 0.0933 0.1465 -0.1894 0.3854 veracityC:educationZ 0.0644 0.1431 -0.2178 0.3425 veracityC:godZ -0.3135 0.1433 -0.6016 -0.0426 veracityC:hireZ -0.1536 0.1390 -0.4276 0.1189 # no interaction repetitionC:ageZ 0.0488 0.0800 -0.1083 0.2059 repetitionC:genderZ 0.0460 0.0833 -0.1173 0.2105 repetitionC:aotZ -0.1165 0.0943 -0.3080 0.0621 repetitionC:covidconcernZ 0.0079 0.0846 -0.1585 0.1751 repetitionC:sesZ -0.1004 0.0823 -0.2604 0.0615 repetitionC:attentionZ 0.0177 0.0895 -0.1548 0.1943 repetitionC:educationZ -0.1016 0.0862 -0.2717 0.0694 repetitionC:godZ 0.0887 0.0839 -0.0756 0.2517 repetitionC:hireZ -0.0364 0.0861 -0.2062 0.1334 # no interaction ``` ## three-way interaction model ```r # outcome coded 1 to 6 Estimate Est.Error Q2.5 Q97.5 Intercept 3.3301 0.0787 3.1758 3.4857 veracityC 0.6580 0.1031 0.4566 0.8622 # repetitionC 0.0955 0.0276 0.0417 0.1500 # ageZ -0.0690 0.0719 -0.2097 0.0720 genderZ 0.1380 0.0713 -0.0033 0.2746 aotZ -0.4606 0.0746 -0.6072 -0.3149 covidconcernZ -0.1072 0.0764 -0.2554 0.0441 sesZ 0.1535 0.0706 0.0148 0.2921 # attentionZ -0.1260 0.0769 -0.2750 0.0244 educationZ -0.0392 0.0728 -0.1821 0.1040 godZ 0.1516 0.0742 0.0075 0.2972 # hireZ 0.0798 0.0696 -0.0567 0.2167 veracityC:repetitionC -0.0216 0.0533 -0.1256 0.0828 veracityC:ageZ 0.0383 0.0623 -0.0839 0.1605 veracityC:genderZ -0.0773 0.0610 -0.1966 0.0405 veracityC:aotZ 0.3271 0.0628 0.2040 0.4502 # veracityC:covidconcernZ 0.2363 0.0667 0.1047 0.3661 # veracityC:sesZ -0.0767 0.0615 -0.1958 0.0446 veracityC:attentionZ 0.0750 0.0657 -0.0534 0.2036 veracityC:educationZ 0.0764 0.0616 -0.0452 0.1957 veracityC:godZ -0.0403 0.0638 -0.1659 0.0840 veracityC:hireZ -0.0810 0.0599 -0.1994 0.0348 repetitionC:ageZ 0.0193 0.0281 -0.0360 0.0742 repetitionC:genderZ 0.0170 0.0275 -0.0368 0.0710 repetitionC:aotZ 0.0308 0.0294 -0.0269 0.0881 repetitionC:covidconcernZ 0.0341 0.0300 -0.0247 0.0930 repetitionC:sesZ -0.0487 0.0277 -0.1026 0.0059 repetitionC:attentionZ -0.0017 0.0297 -0.0606 0.0566 repetitionC:educationZ 0.0024 0.0284 -0.0535 0.0573 repetitionC:godZ 0.0023 0.0291 -0.0544 0.0595 repetitionC:hireZ -0.0006 0.0277 -0.0551 0.0538 veracityC:repetitionC:ageZ 0.0420 0.0550 -0.0659 0.1500 veracityC:repetitionC:genderZ 0.0448 0.0536 -0.0602 0.1496 veracityC:repetitionC:aotZ 0.0186 0.0564 -0.0917 0.1303 veracityC:repetitionC:covidconcernZ 0.1548 0.0582 0.0388 0.2685 # veracityC:repetitionC:sesZ -0.0606 0.0533 -0.1659 0.0441 veracityC:repetitionC:attentionZ -0.0682 0.0577 -0.1820 0.0446 veracityC:repetitionC:educationZ -0.0689 0.0546 -0.1745 0.0383 veracityC:repetitionC:godZ -0.0460 0.0564 -0.1575 0.0654 veracityC:repetitionC:hireZ 0.0292 0.0535 -0.0758 0.1328 ``` ## false headlines only ```r # outcome coded 1 to 6 Estimate Est.Error Q2.5 Q97.5 Intercept 3.0033 0.0904 2.8248 3.1766 repetitionC 0.1069 0.0328 0.0431 0.1707 # ageZ -0.0892 0.0845 -0.2547 0.0821 genderZ 0.1785 0.0821 0.0140 0.3367 aotZ -0.6264 0.0860 -0.7987 -0.4652 covidconcernZ -0.2322 0.0904 -0.4081 -0.0548 sesZ 0.1938 0.0848 0.0279 0.3590 attentionZ -0.1625 0.0902 -0.3387 0.0136 educationZ -0.0773 0.0840 -0.2425 0.0887 godZ 0.1753 0.0862 0.0077 0.3441 hireZ 0.1220 0.0827 -0.0385 0.2848 repetitionC:ageZ -0.0015 0.0337 -0.0684 0.0652 repetitionC:genderZ -0.0053 0.0334 -0.0715 0.0599 repetitionC:aotZ 0.0216 0.0351 -0.0477 0.0908 repetitionC:covidconcernZ -0.0431 0.0364 -0.1142 0.0298 repetitionC:sesZ -0.0179 0.0331 -0.0834 0.0463 repetitionC:attentionZ 0.0330 0.0355 -0.0360 0.1030 repetitionC:educationZ 0.0375 0.0332 -0.0281 0.1030 repetitionC:godZ 0.0257 0.0352 -0.0434 0.0946 repetitionC:hireZ -0.0158 0.0334 -0.0815 0.0491 ``` ## transform hireZ variable to be more normal ```r # linear model with continuous outcome Estimate Est.Error Q2.5 Q97.5 Intercept 3.3265 0.0798 3.1698 3.4801 veracityC 0.6547 0.1036 0.4515 0.8570 repetitionC 0.0958 0.0275 0.0416 0.1492 ageZ -0.0779 0.0697 -0.2134 0.0573 genderZ 0.1386 0.0700 0.0042 0.2767 aotZ -0.4568 0.0738 -0.5983 -0.3111 covidconcernZ -0.1007 0.0773 -0.2510 0.0505 sesZ 0.1413 0.0686 0.0092 0.2787 attentionZ -0.1312 0.0749 -0.2786 0.0136 educationZ -0.0377 0.0731 -0.1789 0.1051 godZ 0.1641 0.0743 0.0188 0.3105 hireZ 0.1259 0.0693 -0.0073 0.2657 veracityC:repetitionC -0.0206 0.0526 -0.1239 0.0823 veracityC:ageZ 0.0158 0.0595 -0.1011 0.1328 veracityC:genderZ -0.0961 0.0591 -0.2127 0.0204 veracityC:aotZ 0.3265 0.0619 0.2050 0.4475 veracityC:covidconcernZ 0.1827 0.0647 0.0569 0.3101 veracityC:sesZ -0.0515 0.0584 -0.1662 0.0616 veracityC:attentionZ 0.0858 0.0629 -0.0390 0.2113 veracityC:educationZ 0.0953 0.0597 -0.0228 0.2121 veracityC:godZ -0.0290 0.0620 -0.1504 0.0930 veracityC:hireZ -0.0369 0.0585 -0.1514 0.0782 # nothing repetitionC:ageZ 0.0164 0.0278 -0.0384 0.0704 repetitionC:genderZ 0.0147 0.0276 -0.0393 0.0687 repetitionC:aotZ 0.0308 0.0289 -0.0252 0.0874 repetitionC:covidconcernZ 0.0275 0.0306 -0.0332 0.0867 repetitionC:sesZ -0.0462 0.0279 -0.1004 0.0091 repetitionC:attentionZ 0.0005 0.0295 -0.0577 0.0580 repetitionC:educationZ 0.0056 0.0284 -0.0489 0.0616 repetitionC:godZ 0.0051 0.0292 -0.0516 0.0625 repetitionC:hireZ 0.0065 0.0272 -0.0463 0.0600 # nothing # logistic regression with 0/1 outcome Estimate Est.Error Q2.5 Q97.5 Intercept -0.1076 0.2126 -0.5237 0.3095 veracityC 1.3671 0.2396 0.8941 1.8383 repetitionC 0.2946 0.0834 0.1309 0.4595 ageZ -0.2184 0.1985 -0.6096 0.1674 genderZ 0.3418 0.1991 -0.0486 0.7344 aotZ -1.1412 0.2103 -1.5558 -0.7318 covidconcernZ -0.1844 0.2163 -0.6115 0.2420 sesZ 0.3123 0.1993 -0.0748 0.7078 attentionZ -0.4576 0.2137 -0.8833 -0.0430 educationZ -0.0834 0.2004 -0.4831 0.3059 godZ 0.5005 0.2076 0.0865 0.9021 hireZ 0.3969 0.1967 0.0217 0.7881 veracityC:repetitionC -0.1664 0.1557 -0.4691 0.1387 veracityC:ageZ 0.2134 0.1341 -0.0488 0.4786 veracityC:genderZ -0.2563 0.1378 -0.5302 0.0072 veracityC:aotZ 0.9101 0.1560 0.6147 1.2301 veracityC:covidconcernZ 0.4041 0.1448 0.1238 0.6913 veracityC:sesZ -0.1078 0.1366 -0.3783 0.1582 veracityC:attentionZ 0.0990 0.1462 -0.1836 0.3857 veracityC:educationZ 0.0631 0.1438 -0.2167 0.3468 veracityC:godZ -0.3329 0.1429 -0.6202 -0.0599 veracityC:hireZ -0.2079 0.1411 -0.4868 0.0648 # maybe something repetitionC:ageZ 0.0455 0.0793 -0.1112 0.1996 repetitionC:genderZ 0.0465 0.0830 -0.1151 0.2088 repetitionC:aotZ -0.1161 0.0959 -0.3086 0.0660 repetitionC:covidconcernZ 0.0052 0.0842 -0.1616 0.1696 repetitionC:sesZ -0.0973 0.0819 -0.2581 0.0648 repetitionC:attentionZ 0.0176 0.0892 -0.1563 0.1925 repetitionC:educationZ -0.1011 0.0862 -0.2711 0.0672 repetitionC:godZ 0.0864 0.0838 -0.0768 0.2545 repetitionC:hireZ -0.0295 0.0863 -0.1978 0.1417 # nothing ``` # plot ![[results 1.png]] # proportion rated accurate/true ```r veracity repetition prop_rated_accurate 1: f new 0.3627513 2: f repeated 0.4013819 # 10.64935% increase (false headlines) 3: t new 0.5430276 4: t repeated 0.5684673 # 4.684789% increase (true headlines) ``` # random assignment looks good ![[1676952436.png]]