MergeCeption-7B-v2-Nous.md

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
MergeCeption-7B-v2	42.38	67.83	66.6	40.94	54.44

AGIEval

Task	Version	Metric	Value		Stderr
agieval_aqua_rat	0	acc	22.05	±	2.61
		acc_norm	23.62	±	2.67
agieval_logiqa_en	0	acc	31.64	±	1.82
		acc_norm	34.25	±	1.86
agieval_lsat_ar	0	acc	28.26	±	2.98
		acc_norm	24.35	±	2.84
agieval_lsat_lr	0	acc	48.24	±	2.21
		acc_norm	49.80	±	2.22
agieval_lsat_rc	0	acc	62.08	±	2.96
		acc_norm	62.83	±	2.95
agieval_sat_en	0	acc	73.30	±	3.09
		acc_norm	71.84	±	3.14
agieval_sat_en_without_passage	0	acc	39.32	±	3.41
		acc_norm	37.38	±	3.38
agieval_sat_math	0	acc	38.64	±	3.29
		acc_norm	35.00	±	3.22

Average: 42.38%

GPT4All

Task	Version	Metric	Value		Stderr
arc_challenge	0	acc	43.69	±	1.45
		acc_norm	50.68	±	1.46
arc_easy	0	acc	58.63	±	1.01
		acc_norm	56.23	±	1.02
boolq	1	acc	83.73	±	0.65
hellaswag	0	acc	70.84	±	0.45
		acc_norm	83.43	±	0.37
openbookqa	0	acc	35.60	±	2.14
		acc_norm	46.20	±	2.23
piqa	0	acc	73.78	±	1.03
		acc_norm	74.97	±	1.01
winogrande	0	acc	79.56	±	1.13

Average: 67.83%

TruthfulQA

Task	Version	Metric	Value		Stderr
truthfulqa_mc	1	mc1	43.21	±	1.73
		mc2	66.60	±	1.60

Average: 66.6%

Bigbench

Task	Version	Metric	Value		Stderr
bigbench_causal_judgement	0	multiple_choice_grade	55.79	±	3.61
bigbench_date_understanding	0	multiple_choice_grade	45.53	±	2.60
bigbench_disambiguation_qa	0	multiple_choice_grade	55.81	±	3.10
bigbench_geometric_shapes	0	multiple_choice_grade	24.79	±	2.28
		exact_str_match	0.00	±	0.00
bigbench_logical_deduction_five_objects	0	multiple_choice_grade	28.40	±	2.02
bigbench_logical_deduction_seven_objects	0	multiple_choice_grade	19.14	±	1.49
bigbench_logical_deduction_three_objects	0	multiple_choice_grade	48.00	±	2.89
bigbench_movie_recommendation	0	multiple_choice_grade	27.40	±	2.00
bigbench_navigate	0	multiple_choice_grade	50.00	±	1.58
bigbench_reasoning_about_colored_objects	0	multiple_choice_grade	27.80	±	1.00
bigbench_ruin_names	0	multiple_choice_grade	53.79	±	2.36
bigbench_salient_translation_error_detection	0	multiple_choice_grade	34.07	±	1.50
bigbench_snarks	0	multiple_choice_grade	75.69	±	3.20
bigbench_sports_understanding	0	multiple_choice_grade	72.31	±	1.43
bigbench_temporal_sequences	0	multiple_choice_grade	29.90	±	1.45
bigbench_tracking_shuffled_objects_five_objects	0	multiple_choice_grade	22.88	±	1.19
bigbench_tracking_shuffled_objects_seven_objects	0	multiple_choice_grade	17.54	±	0.91
bigbench_tracking_shuffled_objects_three_objects	0	multiple_choice_grade	48.00	±	2.89

Average: 40.94%

Average score: 54.44%

Elapsed time: 01:41:42