Skip to content

Instantly share code, notes, and snippets.

View codelion's full-sized avatar
🎯
Locked in!

Asankhaya Sharma codelion

🎯
Locked in!
View GitHub Profile
@codelion
codelion / memory.py
Created April 13, 2025 01:15
A simple implementation of memory for LLMs
import numpy as np
from sentence_transformers import SentenceTransformer
import anthropic
from typing import List, Dict, Optional
import logging
from dataclasses import dataclass
import uuid
logger = logging.getLogger(__name__)
Model ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K Average
mera-mix-4x7B-v1 46.67 69.64 39.65 46.78 68.27 2.73 45.62

ARC

Task Version Metric Value Stderr
arc_challenge 1 acc,none 0.44
acc_stderr,none 0.01
acc_norm,none 0.47
Model ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K Average
mera-mix-4x7B 72.01 88.82 63.67 77.45 84.61 71.65 76.37

ARC

Task Version Metric Value Stderr
arc_challenge 1 acc,none 0.70
acc_stderr,none 0.01
acc_norm,none 0.72
Model ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K
mera-mix-4x7B 65.7 84.73 Error: File does not exist 51.03 79.48 66.34

ARC

Task Version Metric Value Stderr
arc_challenge 1 acc,none 0.62
acc_stderr,none 0.01
acc_norm,none 0.66
Model ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K
mera-mix-4x7B Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist

ARC

Average: Error: File does not exist%

HellaSwag

Model ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K
mera-mix-4x7B Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist Error: File does not exist

ARC

Average: Error: File does not exist%

HellaSwag

@codelion
codelion / listlengthv2.c
Created June 14, 2017 07:59
Verifying shape property for modified version
struct node{
int val;
struct node* next;
};
/*@
list<> == self=null or self::node<_,q> * q::list<>;
*/
int length(struct node* p)
@codelion
codelion / listlength.c
Last active June 14, 2017 07:21
Verifying shape property
struct node{
int val;
struct node* next;
};
/*@
list<> == self=null or self::node<_,q> * q::list<>;
*/
int length(struct node* p)
default-src 'self'; font-src 'self' https://maxcdn.bootstrapcdn.com; form-action 'self'; frame-ancestors 'none'; report-uri https://report-uri.io/report/camjackson; script-src 'self' https://oss.maxcdn.com; style-src 'self' https://maxcdn.bootstrapcdn.com https://cdn.jsdelivr.net
@codelion
codelion / Main.java
Last active September 7, 2015 02:46 — forked from jsyeo/Main.java
Reflection
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
public class Reflection {
public static void vulnerableMethod() {
}
public void reflectVulnerableMethod() throws ClassNotFoundException, NoSuchMethodException, SecurityException,