Skip to content

Instantly share code, notes, and snippets.

View bigsnarfdude's full-sized avatar

BigsnarfDude bigsnarfdude

View GitHub Profile
import os
from openai import OpenAI
import prompt
def truncate_words(input_string, max_words):
words = input_string.split()
truncated_words = words[:max_words]
return ' '.join(truncated_words)

This text appears to be a summary of a lecture or discussion focused on algebraic geometry, specifically dealing with complex varieties, schemes, and toric degenerations. Here's a breakdown of the key points:

  1. Complex Varieties: The discussion primarily deals with algebraic varieties over complex numbers. However, it acknowledges that extending beyond the complex case is possible for those interested.

  2. General Schemes and Spec of a Ring: The lecture isn't focused on general schemes but rather on specific cases, like the spectrum (Spec) of some ring in variables defined by an ideal. This is related to the concept of vanishing sets of ideals, i.e., sets of points where functions in the ideal vanish.

  3. Vanishing Sets in (\mathbb{C}^n) and (\mathbb{C}^{n+1}): The speaker describes constructing varieties as vanishing sets of ideals within (\mathbb{C}^n) and (\mathbb{C}^{n+1}). They mention removing the zero vector from these sets and dividing by scaling, referring to a projective space

@bigsnarfdude
bigsnarfdude / clean_transcripts.py
Last active February 23, 2024 23:20
process_transcripts.py
import os
import sys
def clean_duplicates(file_name):
duplicates = []
cleaned = []
with open(file_name, 'r') as f:
sentences = f.readlines()
for s in sentences:
if s in cleaned:
@bigsnarfdude
bigsnarfdude / l.py
Created February 16, 2024 14:42
llama_cpp_python using metal gpu and without simple QA python
from llama_cpp import Llama
llm = Llama(
model_path="/Users/vincent/development/llama.cpp/models/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
n_gpu_layers=-1, # Uncomment to use GPU acceleration
seed=1337,
n_ctx=2048)
output = llm(
"[INST] Question: Write a paper on the industrial revolution. Answer: [/INST]", # Prompt
max_tokens=None, # Generate up to 32 tokens, set to None to generate up to the end of the context window
#stop=["Q:", "\n"], # Stop generating just before the model would generate a new question
@bigsnarfdude
bigsnarfdude / clean_transcripts_function.py
Last active February 14, 2024 17:45
clean_transcripts_function.py
import os
import sys
def clean_duplicates(file_name):
duplicates = []
cleaned = []
with open(file_name, 'r') as f:
sentences = f.readlines()
for s in sentences:
if s in cleaned:
@bigsnarfdude
bigsnarfdude / build.md
Last active February 14, 2024 02:56
build.md

Ubuntu 22.04 for Deep Learning

This gist contains steps to setup Ubuntu 22.04 for deep learning.


Install Ubuntu 22.04

  • Computer name: virus
  • Name: thadude
@bigsnarfdude
bigsnarfdude / notes.md
Last active February 14, 2024 03:02
whisper.cpp ggml-large-v3
./main -m models/ggml-large-v3.bin -f ./samples/202206020900_goncalves.wav -otxt  (260s --- 4070 ti super)
./main -m models/ggml-large-v3.bin -f ./wav_files/*.wav -otxt

./main -m models/ggml-large-v3.bin -f ./wavz/*.wav -otxt > ~/wavez.log 2>&1 &
cat wavez.log | grep "output_txt: saving output to"

./main -m ./models/ggml-large-v3.bin -f ~/development/videoSummarization/files/audio/202206020900_goncalves.wav (1093s ---m2 pro)
@bigsnarfdude
bigsnarfdude / merge_kit_example.ipynb
Created February 9, 2024 14:54
merge_kit_example.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@bigsnarfdude
bigsnarfdude / local_llm.py
Last active February 15, 2024 19:21
local_llm.py using ollama on mac mps metal
'''
self host an inference server with basic rag support:
- get a Mac Mini or Mac Studio - just run ollama serve, - run ollama web-ui in docker - add some coding assitant model from ollamahub with the web-ui - upload your documents in the web-ui
No code needed, you have your self hosted LLM with basic RAG giving you answers with your documents in context. For us the deepseek coder 33b model is fast enough on a Mac Studio with 64gb ram and can give pretty good suggestions based on our internal coding documentation.
'''
from openai import OpenAI
client = OpenAI(
@bigsnarfdude
bigsnarfdude / splytter.py
Last active February 8, 2024 22:18
splytter.py audio splitter
from pydub import AudioSegment
from openai import OpenAI
def split_audio_into_chunks(filename, chunk_duration=600000):
audio = AudioSegment.from_file(filename)
chunk_length = len(audio)
start = 0
chunk_filenames = []
i = 0