- [prereg](https://docs.google.com/document/d/1ZfBqYoWmYTlnX3Ef1L9eWQa87R8AVY7595A2nPmKXqE/edit) - condition: treatment, control (dog cat) - models: gpt4-turbo, perplexity llama - interesting chat histories: pid: 1684 (messing around), 1656, 1438 (aging), 763 (decline), 301 (slow) # descriptives ```r # condition/model assignment condition model N <char> <char> <int> 1: control gpt 179 2: control perplexity 210 3: treatment gpt 560 4: treatment perplexity 530 # dem vs rep composition pol demrep N <int> <num> <int> 1: 1 1.0 357 2: 2 1.0 418 3: 3 1.0 215 4: 4 1.5 25 5: 5 2.0 155 6: 6 2.0 215 7: 7 2.0 94 # top issues topissue N prop <char> <int> <num> 1: Economy 429 0.271518987 2: Democracy and electi... 181 0.114556962 3: Abortion rights 176 0.111392405 4: Immigration 122 0.077215190 5: Climate change 93 0.058860759 6: Supreme court and ju... 88 0.055696203 7: Healthcare 84 0.053164557 8: Leadership style 69 0.043670886 9: Racial equity and so... 61 0.038607595 10: Social security and ... 54 0.034177215 11: Foreign policy and i... 40 0.025316456 12: Gun policy 38 0.024050633 13: LGBTQ rights 32 0.020253165 14: Crime 29 0.018354430 15: Role of federal gove... 28 0.017721519 16: Education 25 0.015822785 17: Taxation 15 0.009493671 18: Clean energy 10 0.006329114 19: Cybersecurity and fo... 6 0.003797468 # vote vote N <char> <int> 1: Any one but the abov... 1 2: Anyone but Joe or Tr... 1 3: Chase Oliver 2 4: Chris Christie 1 5: Cornel West 3 6: Dem: Kamala Harris 1 7: Donald Trump 441 8: I don't know yet, bu... 1 9: I would not vote for... 16 10: I would not vote out... 55 11: I would not vote, bu... 40 12: Jill Stein 6 13: Jo Jorgensen 1 14: Joe Biden 863 15: Kennedy 7 16: Michelle Obama 1 17: Nikki Haley 1 18: Prefer not to say 12 19: RFK 3 20: RFK JR 1 21: RFK Jr 1 22: RFK Junior 1 23: Robert F Kennedy 1 24: Robert F Kennedy Jr 1 25: Robert F. Kennedy Jr... 1 26: Robert KENNEDY 1 27: Robert Kennedy 5 28: Robert Kennedy Jr. 1 29: Robert Kennedy, Jr 1 30: Travis Kelce for Pre... 1 31: West or Stein 1 32: anyone but Biden and... 1 33: anyone else 2 34: kennedy 2 35: robert kennedy 1 36: robert kennedy Jr. 1 vote N ``` # results - conditionD: 0 (control), treatment (1) - modelZ: zscore (negative: gpt, positive: perplexity) - variables with `Z` suffix are z-scored - outcome variables are not centered/z-scored ## mean of variables for pre (t1) and post (t2) measures ```r variable condition t1 t2 <char> <char> <num> <num> 1: ai_general control 51.94344 53.79639 2: ai_general treatment 52.70459 54.61651 3: lean_bidentrump control 38.31877 36.96144 4: lean_bidentrump treatment 34.54862 33.94312 5: therm_dem control 61.44730 62.60411 6: therm_dem treatment 62.98899 64.50734 7: therm_repub control 41.06684 40.87661 8: therm_repub treatment 39.81651 39.50459 9: vote_chance control 86.78149 87.35476 10: vote_chance treatment 86.43028 87.60367 variable condition model t1 t2 <char> <char> <char> <num> <num> 1: ai_general control gpt 55.70391 59.36313 2: ai_general control perplexity 48.73810 49.02871 3: ai_general treatment gpt 53.47679 56.08393 4: ai_general treatment perplexity 51.88868 53.06604 5: lean_bidentrump control gpt 38.10615 36.64246 6: lean_bidentrump control perplexity 38.50000 37.23333 7: lean_bidentrump treatment gpt 31.47679 30.44643 8: lean_bidentrump treatment perplexity 37.79434 37.63774 9: therm_dem control gpt 64.93855 65.69832 10: therm_dem control perplexity 58.47143 59.96667 11: therm_dem treatment gpt 65.47857 67.28929 12: therm_dem treatment perplexity 60.35849 61.56792 13: therm_repub control gpt 41.38547 41.18436 14: therm_repub control perplexity 40.79524 40.61429 15: therm_repub treatment gpt 37.82679 37.30357 16: therm_repub treatment perplexity 41.91887 41.83019 17: vote_chance control gpt 86.62570 87.42458 18: vote_chance control perplexity 86.91429 87.29524 19: vote_chance treatment gpt 85.13036 86.28036 20: vote_chance treatment perplexity 87.80377 89.00189 variable condition model t1 t2 ``` ![[1720718041.png]] ## biden_trump_preference (lean) We will test whether the treatment changes preferences for Biden/Trump using this linear regression model: - `post_biden_trump_preference ~ condition * zscore(model) * zscore(pre_biden_trump_preference)` ```r > summary(feols(lean_bidentrump_2 ~ conditionD * modelZ * lean_bidentrump_1Z, data = d0)) OLS estimation, Dep. Var.: lean_bidentrump_2 Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 34.641427 0.465404 74.433024 < 2.2e-16 *** conditionD 0.687376 0.542136 1.267903 0.20503 modelZ 0.079641 0.465805 0.170975 0.86427 lean_bidentrump_1Z 40.230393 0.457563 87.923171 < 2.2e-16 *** conditionD:modelZ 0.409874 0.542493 0.755538 0.45005 conditionD:lean_bidentrump_1Z 0.277684 0.537053 0.517051 0.60520 modelZ:lean_bidentrump_1Z 0.400809 0.457847 0.875422 0.38149 conditionD:modelZ:lean_bidentrump_1Z -0.083294 0.537377 -0.155001 0.87684 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 9.1161 Adj. R2: 0.951073 ``` ## voting likelihood We will test whether the treatment changes voting likelihood using this linear regression model: - `post_vote_likelihood ~ condition * zscore(model) * zscore(pre_biden_trump_preference) + zscore(pre_vote_likelihood)` ```r > summary(feols(vote_chance_2 ~ conditionD * modelZ * lean_bidentrump_1Z + vote_chance_1Z, data = d0)) OLS estimation, Dep. Var.: vote_chance_2 Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 87.173831 0.378521 230.300909 < 2.2e-16 *** conditionD 0.575287 0.440942 1.304676 0.19221 modelZ -0.200092 0.378844 -0.528165 0.59746 lean_bidentrump_1Z 0.139939 0.372195 0.375983 0.70698 vote_chance_1Z 25.106534 0.193253 129.915656 < 2.2e-16 *** conditionD:modelZ 0.299737 0.441338 0.679155 0.49715 conditionD:lean_bidentrump_1Z -0.027845 0.436812 -0.063747 0.94918 modelZ:lean_bidentrump_1Z -0.012241 0.372477 -0.032864 0.97379 conditionD:modelZ:lean_bidentrump_1Z 0.063900 0.437099 0.146191 0.88379 ``` ## ai trust ```r > summary(feols(ai_general_2 ~ conditionD * modelZ * ai_general_1Z, data = d0)) OLS estimation, Dep. Var.: ai_general_2 Observations: 1,478 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 54.471603 0.655412 83.110461 < 2.2e-16 *** conditionD -0.021778 0.761401 -0.028602 0.9771856 modelZ -2.015717 0.655967 -3.072892 0.0021589 ** ai_general_1Z 25.696608 0.667194 38.514443 < 2.2e-16 *** conditionD:modelZ 1.265523 0.761889 1.661033 0.0969201 . conditionD:ai_general_1Z 0.505196 0.773147 0.653427 0.5135830 modelZ:ai_general_1Z 0.594800 0.668424 0.889854 0.3736900 conditionD:modelZ:ai_general_1Z -1.041124 0.774279 -1.344638 0.1789498 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 12.7 Adj. R2: 0.805897 ``` ## thermometer dem ```r > summary(feols(therm_dem_2 ~ conditionD * modelZ * therm_dem_1Z, data = d0)) OLS estimation, Dep. Var.: therm_dem_2 Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 63.327559 0.508452 124.549727 < 2.2e-16 *** conditionD 0.429794 0.591645 0.726439 0.46769 modelZ 0.356321 0.508876 0.700211 0.48391 therm_dem_1Z 28.111110 0.503291 55.854620 < 2.2e-16 *** conditionD:modelZ -0.720303 0.592024 -1.216680 0.22392 conditionD:therm_dem_1Z -0.502644 0.588958 -0.853446 0.39355 modelZ:therm_dem_1Z -0.038474 0.503689 -0.076384 0.93912 conditionD:modelZ:therm_dem_1Z 0.384652 0.589318 0.652707 0.51405 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 9.91015 Adj. R2: 0.885272 ``` ## thermometer repub ```r > summary(feols(therm_repub_2 ~ conditionD * modelZ * therm_repub_1Z, data = d0)) OLS estimation, Dep. Var.: therm_repub_2 Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 40.455892 0.459073 88.125171 < 2.2e-16 *** conditionD -0.123052 0.534770 -0.230102 0.81804 modelZ 0.003093 0.459471 0.006732 0.99463 therm_repub_1Z 32.360773 0.448445 72.162237 < 2.2e-16 *** conditionD:modelZ 0.197193 0.535122 0.368502 0.71255 conditionD:therm_repub_1Z 0.932605 0.526763 1.770446 0.07686 . modelZ:therm_repub_1Z 0.110408 0.448684 0.246072 0.80566 conditionD:modelZ:therm_repub_1Z 0.013613 0.527030 0.025829 0.97940 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 RMSE: 9.00512 Adj. R2: 0.930537 ``` ## exploratory stuff When we predict difference scores (including both control and treatment groups, but ignoring condition)—same as what we did for the exploratory studies—we replicated the pilot effects. ```r # lean more biden > summary(feols(diff_lean_bidentrump ~ 1, data = d0)) OLS estimation, Dep. Var.: diff_lean_bidentrump Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) -0.803245 0.23819 -3.37228 0.00076473 *** # for each condition condition term estimate std.error statistic p.value <char> <char> <num> <num> <num> <num> 1: control (Intercept) -1.3573265 0.5160692 -2.630125 0.008875062 2: treatment (Intercept) -0.6055046 0.2654805 -2.280787 0.022753542 condition model term estimate std.error statistic p.value <char> <char> <char> <num> <num> <num> <num> 1: control gpt (Intercept) -1.4636872 0.8324313 -1.7583279 0.080410360 2: control perplexity (Intercept) -1.2666667 0.6424468 -1.9716288 0.049971290 3: treatment gpt (Intercept) -1.0303571 0.3938192 -2.6163204 0.009128641 4: treatment perplexity (Intercept) -0.1566038 0.3528250 -0.4438567 0.657327653 model vote_bidentrump condition term estimate std.error statistic p.value <char> <fctr> <char> <char> <num> <num> <num> <num> 1: gpt biden control (Intercept) -2.0673 1.2744 -1.6221 0.1078 2: gpt biden treatment (Intercept) -1.3840 0.4121 -3.3582 0.0009 3: gpt trump control (Intercept) -0.1964 1.1053 -0.1777 0.8596 4: gpt trump treatment (Intercept) -0.1643 0.9295 -0.1767 0.8600 5: gpt other control (Intercept) -1.8947 1.5158 -1.2500 0.2273 6: gpt other treatment (Intercept) -1.0000 1.4851 -0.6733 0.5029 7: perplexity biden control (Intercept) -0.8108 0.7202 -1.1258 0.2627 8: perplexity biden treatment (Intercept) -0.8027 0.3751 -2.1397 0.0332 9: perplexity trump control (Intercept) -0.8310 1.2893 -0.6445 0.5214 10: perplexity trump treatment (Intercept) 0.8218 0.5888 1.3957 0.1646 11: perplexity other control (Intercept) -4.1786 2.0721 -2.0165 0.0538 12: perplexity other treatment (Intercept) 0.2456 1.9083 0.1287 0.8981 # more likely to vote > summary(feols(diff_vote_chance ~ 1, data = d0)) OLS estimation, Dep. Var.: diff_vote_chance Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 1.01555 0.197734 5.13594 3.1813e-07 *** condition term estimate std.error statistic p.value <char> <char> <num> <num> <num> <num> 1: control (Intercept) 0.5732648 0.2861798 2.003163 4.585487e-02 2: treatment (Intercept) 1.1733945 0.2479972 4.731482 2.521863e-06 condition model term estimate std.error statistic p.value <char> <char> <char> <num> <num> <num> <num> 1: control gpt (Intercept) 0.7988827 0.4764480 1.676747 0.0953473151 2: control perplexity (Intercept) 0.3809524 0.3412655 1.116293 0.2655783873 3: treatment gpt (Intercept) 1.1500000 0.3595001 3.198886 0.0014577355 4: treatment perplexity (Intercept) 1.1981132 0.3407096 3.516523 0.0004748081 model vote_bidentrump condition term estimate std.error statistic p.value <char> <fctr> <char> <char> <num> <num> <num> <num> 1: gpt biden control (Intercept) 0.6250 0.5505 1.1353 0.2589 2: gpt biden treatment (Intercept) 0.8281 0.4119 2.0104 0.0452 3: gpt trump control (Intercept) 0.3036 0.4529 0.6702 0.5055 4: gpt trump treatment (Intercept) 1.2714 0.7937 1.6020 0.1114 5: gpt other control (Intercept) 3.2105 3.0718 1.0452 0.3098 6: gpt other treatment (Intercept) 2.4930 1.2225 2.0392 0.0452 7: perplexity biden control (Intercept) 0.3243 0.4538 0.7147 0.4763 8: perplexity biden treatment (Intercept) 1.3077 0.4363 2.9974 0.0030 9: perplexity trump control (Intercept) 0.3239 0.3621 0.8947 0.3740 10: perplexity trump treatment (Intercept) 1.2586 0.4114 3.0592 0.0026 11: perplexity other control (Intercept) 0.7500 1.6024 0.4680 0.6435 12: perplexity other treatment (Intercept) 0.4386 1.8097 0.2424 0.8094 # trust ai more > summary(feols(diff_ai ~ 1, data = d0)) OLS estimation, Dep. Var.: diff_ai Observations: 1,478 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 1.8613 0.33507 5.55495 3.2887e-08 *** condition term estimate std.error statistic p.value <char> <char> <num> <num> <num> <num> 1: control (Intercept) 1.719072 0.5760475 2.984254 3.023058e-03 2: treatment (Intercept) 1.911927 0.4055760 4.714102 2.742224e-06 condition model term estimate std.error statistic p.value <char> <char> <char> <num> <num> <num> <num> 1: control gpt (Intercept) 3.65921788 0.8512161 4.29881191 2.823321e-05 2: control perplexity (Intercept) 0.05741627 0.7657275 0.07498264 9.403006e-01 3: treatment gpt (Intercept) 2.60714286 0.5343299 4.87927539 1.390664e-06 4: treatment perplexity (Intercept) 1.17735849 0.6129039 1.92095118 5.527499e-02 model vote_bidentrump condition term estimate std.error statistic p.value <char> <fctr> <char> <char> <num> <num> <num> <num> 1: gpt biden control (Intercept) 3.7019 0.7958 4.6519 0.0000 2: gpt biden treatment (Intercept) 4.0716 0.6499 6.2646 0.0000 3: gpt trump control (Intercept) 3.8750 2.1013 1.8441 0.0706 4: gpt trump treatment (Intercept) 1.5357 1.0148 1.5134 0.1325 5: gpt other control (Intercept) 2.7895 2.8020 0.9955 0.3327 6: gpt other treatment (Intercept) -2.4789 1.7537 -1.4135 0.1619 7: perplexity biden control (Intercept) 0.6486 1.1769 0.5511 0.5827 8: perplexity biden treatment (Intercept) 3.3913 0.7741 4.3810 0.0000 9: perplexity trump control (Intercept) -0.1286 0.9464 -0.1359 0.8923 10: perplexity trump treatment (Intercept) -1.5402 1.1188 -1.3766 0.1704 11: perplexity other control (Intercept) -1.8214 2.3317 -0.7812 0.4415 12: perplexity other treatment (Intercept) -2.1404 1.8454 -1.1598 0.2511 # like dem more > summary(feols(diff_therm_dem ~ 1, data = d0)) OLS estimation, Dep. Var.: diff_therm_dem Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) 1.42326 0.258403 5.50791 4.2766e-08 *** condition term estimate std.error statistic p.value <char> <char> <num> <num> <num> <num> 1: control (Intercept) 1.156812 0.4637885 2.494267 1.303702e-02 2: treatment (Intercept) 1.518349 0.3091652 4.911124 1.044133e-06 condition model term estimate std.error statistic p.value <char> <char> <char> <num> <num> <num> <num> 1: control gpt (Intercept) 0.7597765 0.7229019 1.051009 2.946791e-01 2: control perplexity (Intercept) 1.4952381 0.5992799 2.495058 1.336775e-02 3: treatment gpt (Intercept) 1.8107143 0.4397218 4.117863 4.401703e-05 4: treatment perplexity (Intercept) 1.2094340 0.4340920 2.786124 5.525715e-03 model vote_bidentrump condition term estimate std.error statistic p.value <char> <fctr> <char> <char> <num> <num> <num> <num> 1: gpt biden control (Intercept) 0.5962 1.0056 0.5928 0.5546 2: gpt biden treatment (Intercept) 2.2980 0.4717 4.8714 0.0000 3: gpt trump control (Intercept) 1.1607 1.2413 0.9351 0.3538 4: gpt trump treatment (Intercept) 1.5143 1.2255 1.2357 0.2187 5: gpt other control (Intercept) 0.4737 1.7595 0.2692 0.7908 6: gpt other treatment (Intercept) 0.0000 0.8913 0.0000 1.0000 7: perplexity biden control (Intercept) 2.6036 0.8791 2.9616 0.0038 8: perplexity biden treatment (Intercept) 2.2676 0.5096 4.4501 0.0000 9: perplexity trump control (Intercept) 0.2535 1.0620 0.2387 0.8120 10: perplexity trump treatment (Intercept) -0.6552 0.9256 -0.7078 0.4800 11: perplexity other control (Intercept) 0.2500 0.7553 0.3310 0.7432 12: perplexity other treatment (Intercept) 1.3509 0.9638 1.4016 0.1665 # DID NOT like republicans more > summary(feols(diff_therm_repub ~ 1, data = d0)) OLS estimation, Dep. Var.: diff_therm_repub Observations: 1,479 Standard-errors: IID Estimate Std. Error t value Pr(>|t|) (Intercept) -0.279919 0.234566 -1.19335 0.23292 condition term estimate std.error statistic p.value <char> <char> <num> <num> <num> <num> 1: control (Intercept) -0.1902314 0.4775866 -0.398318 0.6906148 2: treatment (Intercept) -0.3119266 0.2689228 -1.159911 0.2463392 condition model term estimate std.error statistic p.value <char> <char> <char> <num> <num> <num> <num> 1: control gpt (Intercept) -0.20111732 0.7213391 -0.2788110 0.7807135 2: control perplexity (Intercept) -0.18095238 0.6376778 -0.2837677 0.7768693 3: treatment gpt (Intercept) -0.52321429 0.3678549 -1.4223389 0.1554856 4: treatment perplexity (Intercept) -0.08867925 0.3935893 -0.2253091 0.8218259 model vote_bidentrump condition term estimate std.error statistic p.value <char> <fctr> <char> <char> <num> <num> <num> <num> 1: gpt biden control (Intercept) -0.9423 1.0318 -0.9133 0.3632 2: gpt biden treatment (Intercept) -1.3467 0.5016 -2.6847 0.0076 3: gpt trump control (Intercept) 0.1786 0.9734 0.1835 0.8551 4: gpt trump treatment (Intercept) 1.4286 0.6129 2.3308 0.0212 5: gpt other control (Intercept) 2.7368 2.4349 1.1240 0.2758 6: gpt other treatment (Intercept) -0.3239 0.8646 -0.3747 0.7090 7: perplexity biden control (Intercept) -0.9099 0.6972 -1.3051 0.1946 8: perplexity biden treatment (Intercept) -1.3779 0.4773 -2.8871 0.0042 9: perplexity trump control (Intercept) 1.0845 1.3901 0.7802 0.4379 10: perplexity trump treatment (Intercept) 2.5287 0.7539 3.3542 0.0010 11: perplexity other control (Intercept) -0.5000 1.6797 -0.2977 0.7682 12: perplexity other treatment (Intercept) -1.3158 1.1542 -1.1400 0.2591 ``` ## preregistered models, split by initial vote preference (biden vs trump vs other) ```r # outcome: lean biden vs trump (negative: more biden) vote_bidentrump term estimate std.error statistic p.value sig <fctr> <char> <num> <num> <num> <num> <char> 1: biden conditionD 2.3902 1.3819 1.7297 0.0840 2: biden modelZ 7.1424 1.1604 6.1551 0.0000 * 3: biden lean_bidentrump_1Z 26.8105 1.4772 18.1496 0.0000 * 4: biden conditionD:modelZ -5.6094 1.3806 -4.0630 0.0001 * # perplexity made biden voters lean more biden 5: biden conditionD:lean_bidentrump_1Z 3.1369 1.7609 1.7815 0.0752 6: biden modelZ:lean_bidentrump_1Z 9.2627 1.4752 6.2790 0.0000 * 7: biden conditionD:modelZ:lean_bidentrump_1Z -7.6018 1.7590 -4.3217 0.0000 * # perplexity work well even for biden voters who leaned more trump 8: trump conditionD 3.5084 4.5550 0.7702 0.4416 9: trump modelZ -10.9006 3.8671 -2.8188 0.0050 * 10: trump lean_bidentrump_1Z 34.1486 2.7075 12.6127 0.0000 * 11: trump conditionD:modelZ 10.9512 4.5701 2.3963 0.0170 * # perplexity made trump voters lean more trump 12: trump conditionD:lean_bidentrump_1Z -2.2309 3.2179 -0.6933 0.4885 13: trump modelZ:lean_bidentrump_1Z 7.4759 2.7177 2.7508 0.0062 * 14: trump conditionD:modelZ:lean_bidentrump_1Z -7.1206 3.2278 -2.2060 0.0279 * # the effect above isn't as strong for trump voters who lean more trump 15: other conditionD 3.0315 2.1208 1.4295 0.1547 16: other modelZ -0.9307 1.8338 -0.5075 0.6125 17: other lean_bidentrump_1Z 39.6298 2.9091 13.6227 0.0000 * 18: other conditionD:modelZ 1.4601 2.1237 0.6876 0.4927 19: other conditionD:lean_bidentrump_1Z -4.7062 3.3352 -1.4111 0.1601 20: other modelZ:lean_bidentrump_1Z -3.5130 2.9139 -1.2056 0.2297 21: other conditionD:modelZ:lean_bidentrump_1Z 4.8558 3.3394 1.4541 0.1478 vote_bidentrump term estimate std.error statistic p.value sig # voting likelihood vote_bidentrump term estimate std.error statistic p.value sig <fctr> <char> <num> <num> <num> <num> <char> 1: biden conditionD 1.3051 1.3241 0.9856 0.3246 2: biden modelZ -0.3880 1.1152 -0.3479 0.7280 3: biden lean_bidentrump_1Z -1.0277 1.4379 -0.7147 0.4750 4: biden vote_chance_1Z 22.8309 0.3920 58.2438 0.0000 * 5: biden conditionD:modelZ 0.5397 1.3230 0.4080 0.6834 6: biden conditionD:lean_bidentrump_1Z 0.9127 1.6876 0.5408 0.5888 7: biden modelZ:lean_bidentrump_1Z -0.1091 1.4167 -0.0770 0.9386 8: biden conditionD:modelZ:lean_bidentrump_1Z 0.0752 1.6853 0.0446 0.9644 9: trump conditionD 1.9536 3.1037 0.6295 0.5294 10: trump modelZ 0.5479 2.6443 0.2072 0.8360 11: trump lean_bidentrump_1Z 0.4604 1.8605 0.2475 0.8047 12: trump vote_chance_1Z 24.0145 0.4275 56.1693 0.0000 * 13: trump conditionD:modelZ 2.6257 3.1244 0.8404 0.4012 14: trump conditionD:lean_bidentrump_1Z -0.7888 2.1926 -0.3597 0.7192 15: trump modelZ:lean_bidentrump_1Z -0.2502 1.8566 -0.1348 0.8929 16: trump conditionD:modelZ:lean_bidentrump_1Z -1.9231 2.2067 -0.8715 0.3840 17: other conditionD 0.2935 2.0218 0.1452 0.8847 18: other modelZ -0.0624 1.7565 -0.0355 0.9717 19: other lean_bidentrump_1Z 5.1518 2.7727 1.8581 0.0649 20: other vote_chance_1Z 25.7616 0.6143 41.9359 0.0000 * 21: other conditionD:modelZ -0.8385 2.0249 -0.4141 0.6793 22: other conditionD:lean_bidentrump_1Z -4.4810 3.1845 -1.4071 0.1613 23: other modelZ:lean_bidentrump_1Z -4.7751 2.7787 -1.7185 0.0876 24: other conditionD:modelZ:lean_bidentrump_1Z 7.0292 3.1957 2.1996 0.0292 * vote_bidentrump term estimate std.error statistic p.value sig # trust ai vote_bidentrump term estimate std.error statistic p.value sig <fctr> <char> <num> <num> <num> <num> <char> 1: biden conditionD 1.5628 0.9779 1.5982 0.1104 2: biden modelZ -1.8159 0.8527 -2.1296 0.0335 * 3: biden ai_general_1Z 26.2679 0.8700 30.1913 0.0000 * 4: biden conditionD:modelZ 1.3447 0.9782 1.3747 0.1696 5: biden conditionD:ai_general_1Z -0.6471 0.9966 -0.6492 0.5164 6: biden modelZ:ai_general_1Z -0.1204 0.8713 -0.1382 0.8901 7: biden conditionD:modelZ:ai_general_1Z -0.3817 0.9977 -0.3826 0.7021 8: trump conditionD -1.5840 1.4067 -1.1260 0.2608 9: trump modelZ -1.8358 1.1899 -1.5428 0.1236 10: trump ai_general_1Z 25.5875 1.2110 21.1301 0.0000 * 11: trump conditionD:modelZ 0.3617 1.4083 0.2568 0.7974 12: trump conditionD:ai_general_1Z 1.3378 1.4290 0.9362 0.3497 13: trump modelZ:ai_general_1Z 0.9698 1.2141 0.7988 0.4249 14: trump conditionD:modelZ:ai_general_1Z -1.2368 1.4322 -0.8636 0.3883 15: other conditionD -4.0019 2.4670 -1.6222 0.1066 16: other modelZ -3.3714 2.1210 -1.5896 0.1138 17: other ai_general_1Z 23.2106 2.1427 10.8323 0.0000 * 18: other conditionD:modelZ 3.5001 2.4699 1.4171 0.1583 19: other conditionD:ai_general_1Z 4.0789 2.4755 1.6477 0.1013 20: other modelZ:ai_general_1Z 3.5852 2.1468 1.6700 0.0968 21: other conditionD:modelZ:ai_general_1Z -3.6678 2.4787 -1.4797 0.1408 vote_bidentrump term estimate std.error statistic p.value sig # thermometer dem vote_bidentrump term estimate std.error statistic p.value sig <fctr> <char> <num> <num> <num> <num> <char> 1: biden conditionD 1.2073 0.9027 1.3374 0.1814 2: biden modelZ 0.8901 0.7841 1.1352 0.2566 3: biden therm_dem_1Z 25.7637 0.9585 26.8781 0.0000 * 4: biden conditionD:modelZ -0.8791 0.9043 -0.9721 0.3313 5: biden conditionD:therm_dem_1Z -0.8712 1.1030 -0.7898 0.4299 6: biden modelZ:therm_dem_1Z -0.3689 0.9596 -0.3845 0.7007 7: biden conditionD:modelZ:therm_dem_1Z -0.0941 1.1039 -0.0852 0.9321 8: trump conditionD -1.8727 1.6939 -1.1055 0.2695 9: trump modelZ -1.3808 1.4202 -0.9722 0.3315 10: trump therm_dem_1Z 28.7387 1.0750 26.7344 0.0000 * 11: trump conditionD:modelZ 1.1005 1.6953 0.6492 0.5166 12: trump conditionD:therm_dem_1Z -1.9242 1.2952 -1.4856 0.1381 13: trump modelZ:therm_dem_1Z -1.1231 1.0761 -1.0437 0.2972 14: trump conditionD:modelZ:therm_dem_1Z 2.0485 1.2965 1.5800 0.1148 15: other conditionD 0.3476 1.2698 0.2737 0.7846 16: other modelZ -0.0376 1.0688 -0.0351 0.9720 17: other therm_dem_1Z 28.6115 1.3082 21.8715 0.0000 * 18: other conditionD:modelZ 0.7396 1.2708 0.5820 0.5613 19: other conditionD:therm_dem_1Z -0.1100 1.5012 -0.0733 0.9417 20: other modelZ:therm_dem_1Z 0.3851 1.3112 0.2937 0.7694 21: other conditionD:modelZ:therm_dem_1Z -0.3910 1.5038 -0.2600 0.7952 vote_bidentrump term estimate std.error statistic p.value sig # thermometer republican vote_bidentrump term estimate std.error statistic p.value sig <fctr> <char> <num> <num> <num> <num> <char> 1: biden conditionD 0.1476 0.9265 0.1593 0.8735 2: biden modelZ 0.0377 0.8139 0.0463 0.9631 3: biden therm_repub_1Z 30.2678 0.9018 33.5624 0.0000 * 4: biden conditionD:modelZ -0.2717 0.9259 -0.2935 0.7692 5: biden conditionD:therm_repub_1Z 0.8175 1.0307 0.7932 0.4279 6: biden modelZ:therm_repub_1Z 0.2038 0.9008 0.2263 0.8210 7: biden conditionD:modelZ:therm_repub_1Z -0.5397 1.0297 -0.5241 0.6003 8: trump conditionD 1.9540 2.0432 0.9563 0.3394 9: trump modelZ 2.1312 1.6824 1.2668 0.2059 10: trump therm_repub_1Z 30.5202 1.3392 22.7904 0.0000 * 11: trump conditionD:modelZ -0.3370 2.0469 -0.1646 0.8693 12: trump conditionD:therm_repub_1Z -0.3464 1.6406 -0.2111 0.8329 13: trump modelZ:therm_repub_1Z -1.7386 1.3419 -1.2956 0.1958 14: trump conditionD:modelZ:therm_repub_1Z 0.6544 1.6430 0.3983 0.6906 15: other conditionD -1.8408 1.4871 -1.2379 0.2175 16: other modelZ -1.4971 1.2727 -1.1763 0.2411 17: other therm_repub_1Z 31.5920 1.5774 20.0275 0.0000 * 18: other conditionD:modelZ 1.0218 1.4889 0.6863 0.4935 19: other conditionD:therm_repub_1Z 1.4410 1.8205 0.7915 0.4298 20: other modelZ:therm_repub_1Z 0.5278 1.5827 0.3335 0.7392 21: other conditionD:modelZ:therm_repub_1Z -0.3792 1.8250 -0.2078 0.8356 vote_bidentrump term estimate std.error statistic p.value sig ``` ## figures (95% CI) ![[1720724838.png]] ![[1720724941.png]] ![[1720724995.png]] ![[1720725053.png]] ![[1720725096.png]] ## regressions by vote and model (DV: lean biden or trump [negative: lean more biden]) ### Biden voters When model is **gpt** and for participants who vote **Biden**, the treatment made them lean more Trump (positive `conditionD` effect), and even more so for those who initially leaned more toward Trump (negative `conditionD:lean_bidentrump_1Z` effect). ```r tidy(lm(lean_bidentrump_2 ~ conditionD * lean_bidentrump_1Z, data = d00[vote_bidentrump == "biden" & model == "gpt"])) |> mutate_if(is.numeric, round, 5) term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 17.0 1.57 10.9 0 2 conditionD 7.93 1.90 4.17 0.00004 3 lean_bidentrump_1Z 17.4 1.98 8.80 0 4 conditionD:lean_bidentrump_1Z 10.8 2.42 4.45 0.00001 ``` When model is **perplexity** and for participants who vote **Biden**, the treatment made them lean more Biden (negative `conditionD` effect), and even more so for those who initially leaned more toward Trump (negative `conditionD:lean_bidentrump_1Z` effect). ```r tidy(lm(lean_bidentrump_2 ~ conditionD * lean_bidentrump_1Z, data = d00[vote_bidentrump == "biden" & model == "perplexity"])) |> mutate_if(is.numeric, round, 5) term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 31.1 1.61 19.3 0 2 conditionD -3.13 1.89 -1.66 0.0982 3 lean_bidentrump_1Z 35.9 2.08 17.2 0 4 conditionD:lean_bidentrump_1Z -4.39 2.45 -1.79 0.0736 ``` ### Trump voters When model is **gpt** and for participants who vote **Trump**, there aren't clear effects. ```r tidy(lm(lean_bidentrump_2 ~ conditionD * lean_bidentrump_1Z, data = d00[vote_bidentrump == "trump" & model == "gpt"])) |> mutate_if(is.numeric, round, 5) term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 56.0 7.29 7.68 0 2 conditionD -7.57 8.32 -0.911 0.364 # lean more biden 3 lean_bidentrump_1Z 26.5 5.00 5.31 0 4 conditionD:lean_bidentrump_1Z 4.92 5.75 0.857 0.393 ``` When model is **perplexity** and for participants who vote **Trump**, the treatment made them lean more Trump (positive `conditionD` effect). But for those who leaned more Trump intially, treatment made them lean more Biden (negative `conditionD:lean_bidentrump_1Z` effect). ```r tidy(lm(lean_bidentrump_2 ~ conditionD * lean_bidentrump_1Z, data = d00[vote_bidentrump == "trump" & model == "perplexity"])) |> mutate_if(is.numeric, round, 5) term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 34.0 3.61 9.42 0 2 conditionD 14.5 4.69 3.08 0.00229 3 lean_bidentrump_1Z 41.4 2.62 15.8 0 4 conditionD:lean_bidentrump_1Z -9.26 3.36 -2.76 0.0063 ``` ## regressions by vote and model (DV: vote likelihood) ### Biden voters **Biden** voters and **gpt**. Nothing going on. ```r > tidy(lm(vote_chance_2 ~ conditionD * lean_bidentrump_1Z + vote_chance_1Z, data = d00[vote_bidentrump == "biden" & model == "gpt"])) |> mutate_if(is.numeric, round, 5) # A tibble: 5 × 5 term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 87.6 1.46 60.0 0 2 conditionD 0.799 1.78 0.450 0.653 3 lean_bidentrump_1Z -0.827 1.86 -0.445 0.656 4 vote_chance_1Z 23.2 0.616 37.6 0 5 conditionD:lean_bidentrump_1Z 0.893 2.25 0.396 0.692 ``` **Biden** voters and **perplexity**. Nothing going on. ```r > tidy(lm(vote_chance_2 ~ conditionD * lean_bidentrump_1Z + vote_chance_1Z, data = d00[vote_bidentrump == "biden" & model == "perplexity"])) |> mutate_if(is.numeric, round, 5) # A tibble: 5 × 5 term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 86.7 1.62 53.4 0 2 conditionD 1.83 1.89 0.971 0.332 3 lean_bidentrump_1Z -1.30 2.13 -0.609 0.543 4 vote_chance_1Z 22.8 0.503 45.3 0 5 conditionD:lean_bidentrump_1Z 0.976 2.44 0.399 0.690 ``` ### Trump voters **Trump** voters and **gpt**. Nothing going on. ```r > tidy(lm(vote_chance_2 ~ conditionD * lean_bidentrump_1Z + vote_chance_1Z, data = d00[vote_bidentrump == "trump" & model == "gpt"])) |> mutate_if(is.numeric, round, 5) # A tibble: 5 × 5 term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 86.0 5.83 14.8 0 2 conditionD -0.572 6.61 -0.0866 0.931 3 lean_bidentrump_1Z 0.884 4.00 0.221 0.825 4 vote_chance_1Z 23.9 0.671 35.7 0 5 conditionD:lean_bidentrump_1Z 1.05 4.57 0.230 0.819 ``` **Trump** voters and **perplexity**. They were more likely to vote? ```r > tidy(lm(vote_chance_2 ~ conditionD * lean_bidentrump_1Z + vote_chance_1Z, data = d00[vote_bidentrump == "trump" & model == "perplexity"])) |> mutate_if(is.numeric, round, 5) # A tibble: 5 × 5 term estimate std.error statistic p.value <chr> <dbl> <dbl> <dbl> <dbl> 1 (Intercept) 87.3 1.95 44.9 0 2 conditionD 4.79 2.54 1.88 0.0612 # something here? 3 lean_bidentrump_1Z 0.119 1.42 0.0843 0.933 4 vote_chance_1Z 24.5 0.533 45.9 0 5 conditionD:lean_bidentrump_1Z -2.82 1.82 -1.55 0.123 ```