Loss (robust BIC). Lower better.
Best model: $drift = b0 + b1 \times \text{drift\_perc\_acc}$
```r
> d0[, .(loss = mean(loss, na.rm = T)), keyby = .(model)]
model loss
1: drift_perc_accDR 106.7937 # best model
2: drift_perc_acc 107.3446
3: drift_perc_accDR_concordance 109.6419
```
```r
> m1 <- lmer(loss ~ model + (1| study/id), data = d0)
> summaryh(m1)
term results
1: (Intercept) b = 105.94, SE = 4.07, t(7) = 26.03, p < .001, r = 0.99 # model: drift_perc_acc
2: modeldrift_perc_accDR b = −0.46, SE = 0.10, t(2073) = −4.57, p < .001, r = 0.10 # best model (smallest BIC)
3: modeldrift_perc_accDR_concordance b = −0.40, SE = 0.11, t(2073) = −3.70, p < .001, r = 0.08
> summaryh(anova(m1))
term results
1: model F(2, 2073) = 12.03, p < .001
```
```r
> m2 <- lmer(loss ~ model * platformEC + (1| study/id), data = d0)
> summaryh(m2)
term results
1: (Intercept) b = 105.90, SE = 4.11, t(6) = 25.79, p < .001, r = 1.00
2: modeldrift_perc_accDR b = −0.41, SE = 0.10, t(2071) = −3.98, p < .001, r = 0.09
3: modeldrift_perc_accDR_concordance b = −0.17, SE = 0.13, t(2071) = −1.37, p = .172, r = 0.03
4: platformEC b = 7.01, SE = 8.21, t(6) = 0.85, p = .426, r = 0.33
5: modeldrift_perc_accDR:platformEC b = −0.34, SE = 0.21, t(2071) = −1.63, p = .103, r = 0.04
6: modeldrift_perc_accDR_concordance:platformEC b = −0.89, SE = 0.25, t(2071) = −3.49, p < .001, r = 0.08
> summaryh(anova(m2))
term results
1: model F(2, 2071) = 7.96, p < .001
2: platformEC F(1, 6) = 0.65, p = .452
3: model:platformEC F(2, 2071) = 6.12, p = .002
```
```r
> m3 <- lmer(loss ~ model + (1| platform/study/id), data = d0)
boundary (singular) fit: see ?isSingular
> summaryh(m3)
term results
1: (Intercept) b = 105.94, SE = 4.07, t(7) = 26.03, p < .001, r = 0.99
2: modeldrift_perc_accDR b = −0.46, SE = 0.10, t(2073) = −4.57, p < .001, r = 0.10
3: modeldrift_perc_accDR_concordance b = −0.40, SE = 0.11, t(2073) = −3.70, p < .001, r = 0.08
> summaryh(anova(m3))
term results
1: model F(2, 2073) = 12.03, p < .001
```