Skip to content

Instantly share code, notes, and snippets.

View kwindla's full-sized avatar

Kwindla Hultman Kramer kwindla

24 followers · 0 following
  • United States
View GitHub Profile
@kwindla
kwindla / khk-llama-3-70B.yml
Last active May 28, 2024 04:30
Llama-3-70B config (works on one GPU no deepspeed; OOMs on multiple GPUs during merge)
base_model: meta-llama/Meta-Llama-3-70B
# deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16.json
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
# This will be the path used for the data when it is saved to the Volume in the cloud.
- path: data.jsonl
@kwindla
kwindla / khk-llama-3-70B.yml
Created May 27, 2024 22:54
Llama-3-70B config (OOM) - 2
base_model: meta-llama/Meta-Llama-3-70B
deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16.json
load_in_8bit: true
load_in_4bit: false
strict: false
datasets:
# This will be the path used for the data when it is saved to the Volume in the cloud.
- path: data.jsonl
@kwindla
kwindla / khk-llama-3-70B.yml
Created May 27, 2024 21:56
Llama-3-70B config (OOM)
base_model: meta-llama/Meta-Llama-3-70B
deepspeed: /workspace/axolotl/deepspeed_configs/zero3_bf16.json
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
# This will be the path used for the data when it is saved to the Volume in the cloud.
- path: data.jsonl
@kwindla
kwindla / benchmark-llama3-ttfb.py
Created April 19, 2024 19:41
Groq Llama-3 Time To First Byte
import os
import json
import time
import statistics
from groq import Groq
# Set the Groq API key and the number of inferences to run
GROQ_API_KEY = os.environ.get("GROQ_API_KEY")
NUM_INFERENCES = 10
@kwindla
kwindla / friendly-cartoon-robot.py
Created February 25, 2024 00:58
friendly cartoon robot
from datetime import datetime
import asyncio
import aiohttp
import os
import sys
from dailyai.conversation_wrappers import InterruptibleConversationWrapper
from dailyai.queue_frame import StartStreamQueueFrame, TranscriptionQueueFrame, TextQueueFrame, UserStartedSpeakingFrame, UserStoppedSpeakingFrame
from dailyai.services.daily_transport_service import DailyTransportService
from dailyai.services.fireworks_ai_services import FireworksLLMService
@kwindla
kwindla / minimal-cmd-line-asyncio.py
Created February 13, 2024 15:08
asyncio + daily-python + ctrl-c
#
# minimal daily-python command-line client using asyncio
#
import os
import asyncio
import signal
import time
from daily import *
@kwindla
kwindla / print-transcription-events-asyncio.py
Created February 7, 2024 23:28
daily-python transcription events
import os
import asyncio
import signal
import time
from daily import *
class PrintTranscriptApp(EventHandler):
@kwindla
kwindla / motion-screenshare.html
Last active December 9, 2023 16:33
Daily.co sending a motion-optimized screenshare
<html>
<head>
<title>test high bitrate, motion-oriented screenshare</title>
<script src="https://unpkg.com/@daily-co/daily-js"></script>
</head>
<body onload="main()">
<div id="videos"></div>
<script>
async function main() {
@kwindla
kwindla / video2txt.py
Created November 11, 2023 20:56
Extract text from a video file using GPT-4V (very simple demo script)
#!/usr/bin/env python3
import click
import imageio
import imageio_ffmpeg
import io
import base64
from PIL import Image
from openai import OpenAI
@kwindla
kwindla / hq-video-example.html
Created May 17, 2023 04:00
daily-js: setting aspect ratio, high framerate, high bitrate for camera video track
<html>
<head>
<title>video aspect ratio and high bitrate/framerate</title>
<script src="https://unpkg.com/@daily-co/daily-js"></script>
</head>
<body onload="main()">
<script>
/*
This is sample code showing a simulcast layer configuration