Skip to content

Instantly share code, notes, and snippets.

View tsukumijima's full-sized avatar
📺
DTV

tsukumi tsukumijima

📺
DTV
View GitHub Profile
@tori29umai0123
tori29umai0123 / nsfw_filter.py
Last active May 21, 2024 07:33
nsfw_filter.py
import argparse
import csv
import glob
import os
from pathlib import Path
import cv2
import numpy as np
import torch
from PIL import Image
@thesamesam
thesamesam / xz-backdoor.md
Last active May 31, 2024 14:15
xz-utils backdoor situation (CVE-2024-3094)

FAQ on the xz-utils backdoor (CVE-2024-3094)

This is a living document. Everything in this document is made in good faith of being accurate, but like I just said; we don't yet know everything about what's going on.

Background

On March 29th, 2024, a backdoor was discovered in xz-utils, a suite of software that

@alfredplpl
alfredplpl / gemma_finetune_lora.py
Created February 24, 2024 04:23
Gemma初心者ファインチューニングコードです。HFの設定などはよしなにやってください。
# Reference #1: https://note.com/npaka/n/nc55e44e407ff
# Reference #2: https://huggingface.co/blog/gemma-peft
# Licence: MIT
from peft import LoraConfig
lora_config = LoraConfig(
r=8,
target_modules=["q_proj", "o_proj", "k_proj", "v_proj", "gate_proj", "up_proj", "down_proj"],
task_type="CAUSAL_LM",
[
{
"word": "Asshole",
"kana": "アスホール",
"meaning": "いやな奴(Ass=お尻、Hole=穴)",
"notice": "「うざい野郎」「ろくでなし」"
},
{
"word": "あばずれ",
"kana": "あばずれ",
@yyya-nico
yyya-nico / README.md
Last active May 31, 2024 18:48
KonomiTVですべての地上波チャンネルを同時視聴する

KonomiTVですべての地上波チャンネルを同時視聴する

KonomiTVにファイルを足すことで、地上波多チャンネル同時視聴ができるかもしれない。
当然チューナーは必要数用意
まずフォルダーを以下の場所に作る。

  • {KonomiTVインストールフォルダー}/client/dist/wholech/
  • {KonomiTVインストールフォルダー}/client/dist/wholech/css/
  • {KonomiTVインストールフォルダー}/client/dist/wholech/js/

htmlファイルはwholechフォルダー、cssファイルはcssフォルダー、jsファイルはjsフォルダーに置く。

@subaru-shoji
subaru-shoji / aivis-dataset_and_style-bert-vits2.ipynb
Created January 2, 2024 13:30
aivis-dataset_and_style-bert-vits2.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@litagin02
litagin02 / simple_merge.py
Last active February 25, 2024 16:13
Bert-VITS2のモデルマージするやつ(声音・感情表現それぞれを取っ替えたり混ぜたり)
import os
import gradio as gr
import torch
from infer import get_net_g, infer
import utils
voice_keys = ["dec", "flow"]
speech_style_keys = ["enc_p"]
# MIT License
# This code will run on VRAM 12GB+ GPU such as T4, RTX 3060
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import PyPDFLoader
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.llms.huggingface_pipeline import HuggingFacePipeline
@opparco
opparco / debug-tokenizer-vicuna-13b.py
Created August 21, 2023 04:59
debug tokenizer of lmsys/vicuna-13b-v1.3
#
# debug tokenizer of lmsys/vicuna-13b-v1.3
#
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("lmsys/vicuna-13b-v1.3")
def encode_decode(string: str):
@opparco
opparco / weblab_10b.py
Last active August 20, 2023 10:20
correct vocab of matsuo-lab/weblab-10b
#
# correct vocab of matsuo-lab/weblab-10b
#
vocab = {}
# 95 -> (none)
# 96 -> \xa1
# ...
# 107 -> \xac