burtenshaw

{
  "prog": {
    "lm": null,
    "traces": [],
    "train": [],
    "demos": [
      {
        "augmented": true,
        "question": "Barry has $10.00 worth of dimes.  His little brother, Dan has half that amount but finds 2 more dimes on his way home from school.  How many dimes does Dan have?",
        "rationale": "find out how many dimes Dan has. We know that Barry has $10.00 worth of dimes, and since there are 10 cents in a dollar, we can calculate the number of dimes Barry has as follows:\n\nBarry's dimes = $10.00 * 10 cents\/dollar = 100 dimes\n\nWe also know that Dan has half the amount of dimes that Barry has, so we can find out how many dimes Dan has by calculating:\n\nDan's dimes = Barry's dimes \/ 2 = 100 dimes \/ 2 = 50 dimes\n\nHowever, we",

burtenshaw

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
OpenHermes-2.5-Mistral-7B-top-SPIN-iter0	42.47	72.8	52.58	40.46	52.08

AGIEval

Task	Version	Metric	Value		Stderr
agieval_aqua_rat	0	acc	24.02	±	2.69
		acc_norm	24.02	±	2.69
agieval_logiqa_en	0	acc	38.10	±	1.90