# Idea đź’ˇ
To generate predictions that can be aggregated by the algorithms, we will run a series of prediction surveys. The surveys cover a total of four domains with four questions each and are repeated every month for a total of six months.
Algorithms are domain specific. That is, you can submit up to four different algorithms, one for each domain. Within a domain, however, your algorithm will be applied to each of the four questions of that domain. The algorithm does not know which of the four questions it currently works with. We impose this restriction to prevent overfitting to a specific question.
The goal for each research team is to come up with an algorithm for each domain that aggregates the predictions such that the resulting aggregate prediction is as close as possible to the actual outcome. The algorithms will be evaluated based on their performance on the datasets generated from the prediction surveys.
Input: Your algorithm should load the dataset from `input/dataset.csv`.
Output: Your algorithm should write a plain text file with the final aggregation value to `output/result.txt`.
```r
# load dataset
data <- read.csv("input/dataset.csv")
# take the mean of the prediction column
simple_mean <- mean(data$prediction) # this is where the magic happens
# write result to the output file
write(simple_mean, file="output/result.txt")
```
```python
# package imports
from statistics import mean
import csv
# load dataset
with open('input/dataset.csv', 'r') as f:
reader = csv.DictReader(f)
data = [row for row in reader]
# take the mean of the prediction column
predictions = [float(row['prediction']) for row in data]
simple_mean = mean(predictions) # this is where the magic happens
# write result to the output file
with open('output/result.txt', 'w') as f:
f.write(str(simple_mean))
```
# Tasks and deadlines ⌛
# Brain dump đź§
- baseline model: use predictions from ppl who are fairly accurate on both m1 and m2 predictions (and high confidence?)
- clustering algorithm to identify superforecasters
- ML model like lightGBM to identify superforecasters
- [[240227_002521 superforecaster qualities]]
# Links 🕸️
- [WoC-CAP codebook](https://woccap.com/members/codebook/)
- [WoC-CAP info](https://woccap.com/members/info/)
# Logs and meetings đź“ť
- [[240229_090444 algorithm]]
- [[240226_235826 economics items]]
- [[240226_235805 politics items]]
- [[240226_235602 climate items]]
- [[240226_235834 sports items]]