Mistralpaca-7B-Nous.md

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
Mistralpaca-7B	33.48	70.71	52.89	37.06	48.53

AGIEval

Task	Version	Metric	Value		Stderr
agieval_aqua_rat	0	acc	24.80	±	2.72
		acc_norm	25.98	±	2.76
agieval_logiqa_en	0	acc	28.26	±	1.77
		acc_norm	32.41	±	1.84
agieval_lsat_ar	0	acc	23.91	±	2.82
		acc_norm	20.87	±	2.69
agieval_lsat_lr	0	acc	32.55	±	2.08
		acc_norm	33.14	±	2.09
agieval_lsat_rc	0	acc	42.38	±	3.02
		acc_norm	37.92	±	2.96
agieval_sat_en	0	acc	54.37	±	3.48
		acc_norm	50.97	±	3.49
agieval_sat_en_without_passage	0	acc	38.83	±	3.40
		acc_norm	38.35	±	3.40
agieval_sat_math	0	acc	31.36	±	3.14
		acc_norm	28.18	±	3.04

Average: 33.48%

GPT4All

Task	Version	Metric	Value		Stderr
arc_challenge	0	acc	51.19	±	1.46
		acc_norm	55.12	±	1.45
arc_easy	0	acc	80.47	±	0.81
		acc_norm	76.85	±	0.87
boolq	1	acc	84.22	±	0.64
hellaswag	0	acc	62.89	±	0.48
		acc_norm	81.59	±	0.39
openbookqa	0	acc	34.00	±	2.12
		acc_norm	45.40	±	2.23
piqa	0	acc	79.27	±	0.95
		acc_norm	80.85	±	0.92
winogrande	0	acc	70.96	±	1.28

Average: 70.71%

TruthfulQA

Task	Version	Metric	Value		Stderr
truthfulqa_mc	1	mc1	35.99	±	1.68
		mc2	52.89	±	1.59

Average: 52.89%

Bigbench

Task	Version	Metric	Value		Stderr
bigbench_causal_judgement	0	multiple_choice_grade	56.84	±	3.60
bigbench_date_understanding	0	multiple_choice_grade	67.48	±	2.44
bigbench_disambiguation_qa	0	multiple_choice_grade	30.23	±	2.86
bigbench_geometric_shapes	0	multiple_choice_grade	18.11	±	2.04
		exact_str_match	4.74	±	1.12
bigbench_logical_deduction_five_objects	0	multiple_choice_grade	23.00	±	1.88
bigbench_logical_deduction_seven_objects	0	multiple_choice_grade	17.43	±	1.43
bigbench_logical_deduction_three_objects	0	multiple_choice_grade	38.67	±	2.82
bigbench_movie_recommendation	0	multiple_choice_grade	31.20	±	2.07
bigbench_navigate	0	multiple_choice_grade	50.00	±	1.58
bigbench_reasoning_about_colored_objects	0	multiple_choice_grade	56.15	±	1.11
bigbench_ruin_names	0	multiple_choice_grade	35.71	±	2.27
bigbench_salient_translation_error_detection	0	multiple_choice_grade	23.65	±	1.35
bigbench_snarks	0	multiple_choice_grade	53.59	±	3.72
bigbench_sports_understanding	0	multiple_choice_grade	61.46	±	1.55
bigbench_temporal_sequences	0	multiple_choice_grade	28.00	±	1.42
bigbench_tracking_shuffled_objects_five_objects	0	multiple_choice_grade	21.44	±	1.16
bigbench_tracking_shuffled_objects_seven_objects	0	multiple_choice_grade	15.49	±	0.87
bigbench_tracking_shuffled_objects_three_objects	0	multiple_choice_grade	38.67	±	2.82

Average: 37.06%

Average score: 48.53%

Elapsed time: 02:21:46