Skip to content

Instantly share code, notes, and snippets.

View wassname's full-sized avatar
🙃

Michael J Clark wassname

🙃
295 followers · 170 following
View GitHub Profile
@wassname
wassname / setattrattr.py
Created September 24, 2024 23:03
def setattrattr(cfg, k, v):
"""
Sets an attr even it's like o.a.b
Usage:
setattrattr(obj, 'loss.alpha', 0.1)
https://gist.github.com/wassname/33ef989ab06325daeeef68be1d638a2c/edit
"""
if '.' in k:
@wassname
wassname / goodreads_export.csv
Created September 9, 2024 11:39
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 24 columns, instead of 8 in line 4.
Book Id,Title,Author,Author l-f,Additional Authors,ISBN,ISBN13,My Rating,Average Rating,Publisher,Binding,Number of Pages,Year Published,Original Publication Year,Date Read,Date Added,Bookshelves,Bookshelves with positions,Exclusive Shelf,My Review,Spoiler,Private Notes,Read Count,Owned Copies
60233239,"The Butcher's Masquerade (Dungeon Crawler Carl, #5)",Matt Dinniman,"Dinniman, Matt",,"=""""","=""""",5,4.68,Dandy House,Kindle Edition,726,2022,2022,,2024/08/27,,,read,,,,1,0
12913718,"달빛 조각사 1 (The Legendary Moonlight Sculptor, #1)",Heesung Nam,"Nam, Heesung",,"=""895857903X""","=""9788958579038""",4,4.38,로크미디어,Paperback,344,2007,2007,,2016/11/14,"favs, 5star","favs (#37), 5star (#132)",read,,,,1,0
18527310,"달빛 조각사 4 (The Legendary Moonlight Sculptor, #4)",Heesung Nam,"Nam, Heesung",,"=""8958579064""","=""9788958579062""",4,4.38,로크미디어,Paperback,335,2007,2007,,2024/08/27,,,read,,,,1,0
17661892,"달빛 조각사 2 (The Legendary Moonlight Sculptor, #2)",Heesung Nam,"Nam, Heesung",,"=""8958579048""","=""9788958579045""",4
@wassname
wassname / lighting_save_only_adapter.py
Last active September 6, 2024 03:19
Lightning snippets for use with transformers
from lightning.pytorch.callbacks import ModelCheckpoint
from weakref import proxy
class AdapterModelCheckpoint(ModelCheckpoint):
def _save_checkpoint(self, trainer: "pl.Trainer", filepath: str) -> None:
trainer.model.save_pretrained(filepath)
# trainer.save_checkpoint(filepath, self.save_weights_only)
self._last_global_step_saved = trainer.global_step
self._last_checkpoint_saved = filepath
@wassname
wassname / lightning_bnb_existing.py
Last active September 6, 2024 08:07
DO NOT USE, just set lightning to bfloat16 of float32 instead
# # FIXME DOT NOT USE
# # upon further investigation, it seems the bnb will handle conversion as long as you use bloat16 of float32, float16
# from lightning.pytorch.plugins.precision.precision import Precision
# from lightning.fabric.plugins.precision.utils import (
# _ClassReplacementContextManager,
# _convert_fp_tensor,
# _DtypeContextManager,
# )
# from typing_extensions import Self, override
@wassname
wassname / peft_adapter_is_active.py
Last active August 28, 2024 01:30
Given a peft model work out is adapters are enabled or disabled"
from peft.peft_model import BaseTunerLayer, PeftModel
from peft.utils.other import ModulesToSaveWrapper
def adapter_is_disabled(model: PeftModel) -> bool:
"""Given a peft model work out is adapters are enabled or disabled"""
for module in model.model.modules():
if isinstance(module, (BaseTunerLayer, ModulesToSaveWrapper)):
# print(help(module.enable_adapters))
return module._disable_adapters
@wassname
wassname / symphypothesis.py
Last active August 2, 2024 02:55
fhypothesis: a easy way to display hypothesis in python, kind of like assert
import sympy as sp
from typing import Dict, Any
from IPython.display import display
from sympy import init_printing
init_printing()
def shypothesis(hypothesis: str, variables: Dict[str, Any] = None, round=3, verbose=False):
"""
Evaluate a hypothesis using SymPy, showing simplified equation and result.
@wassname
wassname / craftax_render_symbolic.py
Last active May 24, 2024 23:44
Craftax symbolic to env.state to pixel
"""
craftax = "1.4.1"
jax = "^0.4.28"
jax =
https://gist.github.com/wassname/9f410d11f33cec75393b64d62286dd41
"""
import numpy as np
import numpy as np
from craftax.craftax.craftax_state import EnvState, Inventory, Mobs
from craftax.craftax.constants import MAX_OBS_DIM, OBS_DIM, BlockType, ItemType, MONSTERS_KILLED_TO_CLEAR_LEVEL
@wassname
wassname / launch.json
Created May 19, 2024 02:19
my typical vscode debugger options
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "test",
"type": "debugpy",
"request": "launch",
@wassname
wassname / choice_tree.py
Last active June 3, 2024 13:44
for huggingface transformers sometime you want to constrain output to json schema and record the probabilities on choices/enums. I use it when rating, judging. It's much more efficient than sampling multiple times.
from jaxtyping import Float, Int
import torch
from torch.nn import functional as F
from torch import Tensor
from typing import List, Callable, Tuple, Dict, Optional
import pandas as pd
from transformers import AutoModelForCausalLM, AutoTokenizer
def get_valid_next_choices(choices_tokens, current_tokens):
@wassname
wassname / hf_perplexity.py
Last active December 7, 2024 05:42
simple perplexity for huggingface models similar to llam..cpp
# Directly taken from https://huggingface.co/spaces/evaluate-measurement/perplexity/blob/main/perplexity.py
# TODO replace with a strided version https://github.com/huggingface/transformers/issues/9648#issuecomment-812981524
import numpy as np
import torch
import itertools
from torch.nn import CrossEntropyLoss
from tqdm.auto import tqdm
import torch.nn.functional as F
from datasets import load_dataset, Dataset
from transformers import AutoTokenizer, AutoModelForCausalLM