NeuralTrixlaser-bf16-Nous.md

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
NeuralTrixlaser-bf16	44.43	77	79.33	49.9	62.66

AGIEval

Task	Version	Metric	Value		Stderr
agieval_aqua_rat	0	acc	26.38	±	2.77
		acc_norm	25.98	±	2.76
agieval_logiqa_en	0	acc	38.40	±	1.91
		acc_norm	37.17	±	1.90
agieval_lsat_ar	0	acc	23.48	±	2.80
		acc_norm	23.48	±	2.80
agieval_lsat_lr	0	acc	53.33	±	2.21
		acc_norm	55.10	±	2.20
agieval_lsat_rc	0	acc	64.68	±	2.92
		acc_norm	64.31	±	2.93
agieval_sat_en	0	acc	78.64	±	2.86
		acc_norm	78.16	±	2.89
agieval_sat_en_without_passage	0	acc	41.26	±	3.44
		acc_norm	40.78	±	3.43
agieval_sat_math	0	acc	33.64	±	3.19
		acc_norm	30.45	±	3.11

Average: 44.43%

GPT4All

Task	Version	Metric	Value		Stderr
arc_challenge	0	acc	65.78	±	1.39
		acc_norm	67.92	±	1.36
arc_easy	0	acc	85.35	±	0.73
		acc_norm	79.21	±	0.83
boolq	1	acc	87.22	±	0.58
hellaswag	0	acc	69.52	±	0.46
		acc_norm	87.46	±	0.33
openbookqa	0	acc	39.80	±	2.19
		acc_norm	51.40	±	2.24
piqa	0	acc	82.32	±	0.89
		acc_norm	84.11	±	0.85
winogrande	0	acc	81.69	±	1.09

Average: 77.0%

TruthfulQA

Task	Version	Metric	Value		Stderr
truthfulqa_mc	1	mc1	64.50	±	1.68
		mc2	79.33	±	1.34

Average: 79.33%

Bigbench

Task	Version	Metric	Value		Stderr
bigbench_causal_judgement	0	multiple_choice_grade	61.05	±	3.55
bigbench_date_understanding	0	multiple_choice_grade	60.98	±	2.54
bigbench_disambiguation_qa	0	multiple_choice_grade	59.30	±	3.06
bigbench_geometric_shapes	0	multiple_choice_grade	22.56	±	2.21
		exact_str_match	0.00	±	0.00
bigbench_logical_deduction_five_objects	0	multiple_choice_grade	32.20	±	2.09
bigbench_logical_deduction_seven_objects	0	multiple_choice_grade	23.57	±	1.61
bigbench_logical_deduction_three_objects	0	multiple_choice_grade	62.67	±	2.80
bigbench_movie_recommendation	0	multiple_choice_grade	51.80	±	2.24
bigbench_navigate	0	multiple_choice_grade	56.20	±	1.57
bigbench_reasoning_about_colored_objects	0	multiple_choice_grade	66.80	±	1.05
bigbench_ruin_names	0	multiple_choice_grade	56.70	±	2.34
bigbench_salient_translation_error_detection	0	multiple_choice_grade	40.78	±	1.56
bigbench_snarks	0	multiple_choice_grade	72.93	±	3.31
bigbench_sports_understanding	0	multiple_choice_grade	73.02	±	1.41
bigbench_temporal_sequences	0	multiple_choice_grade	52.10	±	1.58
bigbench_tracking_shuffled_objects_five_objects	0	multiple_choice_grade	23.44	±	1.20
bigbench_tracking_shuffled_objects_seven_objects	0	multiple_choice_grade	19.49	±	0.95
bigbench_tracking_shuffled_objects_three_objects	0	multiple_choice_grade	62.67	±	2.80

Average: 49.9%

Average score: 62.66%

Elapsed time: 01:35:53