Model | AGIEval | GPT4All | TruthfulQA | Bigbench | Average |
---|---|---|---|---|---|
MetaMath-Mistral-7B | 33.91 | 70.12 | 44.83 | 37.71 | 46.64 |
Task | Version | Metric | Value | Stderr | |
---|---|---|---|---|---|
agieval_aqua_rat | 0 | acc | 20.08 | ± | 2.52 |
acc_norm | 23.23 | ± | 2.65 | ||
agieval_logiqa_en | 0 | acc | 31.64 | ± | 1.82 |
acc_norm | 33.95 | ± | 1.86 | ||
agieval_lsat_ar | 0 | acc | 20.00 | ± | 2.64 |
acc_norm | 16.52 | ± | 2.45 | ||
agieval_lsat_lr | 0 | acc | 40.59 | ± | 2.18 |
acc_norm | 37.45 | ± | 2.15 | ||
agieval_lsat_rc | 0 | acc | 48.33 | ± | 3.05 |
acc_norm | 44.98 | ± | 3.04 | ||
agieval_sat_en | 0 | acc | 59.22 | ± | 3.43 |
acc_norm | 54.85 | ± | 3.48 | ||
agieval_sat_en_without_passage | 0 | acc | 39.81 | ± | 3.42 |
acc_norm | 33.50 | ± | 3.30 | ||
agieval_sat_math | 0 | acc | 34.55 | ± | 3.21 |
acc_norm | 26.82 | ± | 2.99 |
Average: 33.91%
Task | Version | Metric | Value | Stderr | |
---|---|---|---|---|---|
arc_challenge | 0 | acc | 47.78 | ± | 1.46 |
acc_norm | 51.19 | ± | 1.46 | ||
arc_easy | 0 | acc | 79.92 | ± | 0.82 |
acc_norm | 77.15 | ± | 0.86 | ||
boolq | 1 | acc | 83.06 | ± | 0.66 |
hellaswag | 0 | acc | 62.73 | ± | 0.48 |
acc_norm | 80.24 | ± | 0.40 | ||
openbookqa | 0 | acc | 32.60 | ± | 2.10 |
acc_norm | 43.60 | ± | 2.22 | ||
piqa | 0 | acc | 80.52 | ± | 0.92 |
acc_norm | 81.66 | ± | 0.90 | ||
winogrande | 0 | acc | 73.95 | ± | 1.23 |
Average: 70.12%
Task | Version | Metric | Value | Stderr | |
---|---|---|---|---|---|
truthfulqa_mc | 1 | mc1 | 29.99 | ± | 1.60 |
mc2 | 44.83 | ± | 1.55 |
Average: 44.83%
Task | Version | Metric | Value | Stderr | |
---|---|---|---|---|---|
bigbench_causal_judgement | 0 | multiple_choice_grade | 55.26 | ± | 3.62 |
bigbench_date_understanding | 0 | multiple_choice_grade | 60.70 | ± | 2.55 |
bigbench_disambiguation_qa | 0 | multiple_choice_grade | 46.51 | ± | 3.11 |
bigbench_geometric_shapes | 0 | multiple_choice_grade | 19.50 | ± | 2.09 |
exact_str_match | 0.00 | ± | 0.00 | ||
bigbench_logical_deduction_five_objects | 0 | multiple_choice_grade | 34.80 | ± | 2.13 |
bigbench_logical_deduction_seven_objects | 0 | multiple_choice_grade | 24.71 | ± | 1.63 |
bigbench_logical_deduction_three_objects | 0 | multiple_choice_grade | 52.00 | ± | 2.89 |
bigbench_movie_recommendation | 0 | multiple_choice_grade | 31.20 | ± | 2.07 |
bigbench_navigate | 0 | multiple_choice_grade | 50.00 | ± | 1.58 |
bigbench_reasoning_about_colored_objects | 0 | multiple_choice_grade | 47.75 | ± | 1.12 |
bigbench_ruin_names | 0 | multiple_choice_grade | 29.02 | ± | 2.15 |
bigbench_salient_translation_error_detection | 0 | multiple_choice_grade | 14.53 | ± | 1.12 |
bigbench_snarks | 0 | multiple_choice_grade | 46.41 | ± | 3.72 |
bigbench_sports_understanding | 0 | multiple_choice_grade | 54.46 | ± | 1.59 |
bigbench_temporal_sequences | 0 | multiple_choice_grade | 21.10 | ± | 1.29 |
bigbench_tracking_shuffled_objects_five_objects | 0 | multiple_choice_grade | 22.56 | ± | 1.18 |
bigbench_tracking_shuffled_objects_seven_objects | 0 | multiple_choice_grade | 16.17 | ± | 0.88 |
bigbench_tracking_shuffled_objects_three_objects | 0 | multiple_choice_grade | 52.00 | ± | 2.89 |
Average: 37.71%
Average score: 46.64%
Elapsed time: 02:01:44