Skip to content

Instantly share code, notes, and snippets.

View wesslen's full-sized avatar

Ryan Wesslen wesslen

View GitHub Profile
@wesslen
wesslen / ceo-letters-chunking.ipynb
Created April 17, 2024 17:03
ceo-letters-chunking.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wesslen
wesslen / fireworks-colabtune-a100.ipynb
Created March 17, 2024 23:47
fireworks-colabtune-a100.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wesslen
wesslen / hf-sft-qlora-mistral7b-t4.ipynb
Last active March 25, 2024 22:20
HF-SFT-QLoRA-Mistral7B-T4.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wesslen
wesslen / pe-openai-lecture.ipynb
Created March 11, 2024 21:57
pe-openai-lecture.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wesslen
wesslen / openai.sql
Last active March 8, 2024 21:24
movie-csv
with query as (
select
openai_embedding(:question, :_cookie_openai_api_key) as q
),
top_n as (
select
value
from json_each(
faiss_search(
'simonwillisonblog',
@wesslen
wesslen / bac-10k-riskfactors.txt
Created March 7, 2024 23:00
bac-10k-riskfactors.txt
The discussion below addresses our material risk factors of which we are aware. Any risk factor, either by itself or together with other risk factors, could materially and adversely affect our businesses, results of operations, cash flows and/or financial condition. References to third parties may include suppliers, service providers, counterparties, financial market utilities, exchanges and clearing houses, data aggregators and other partners and their upstream and downstream service providers (e.g., fourth parties, fifth parties) who may also contribute to our risks. Other factors not currently known to us or that we currently deem immaterial could also adversely affect our businesses, results of operations, cash flows and/or financial condition. Therefore, the risk factors below should not be considered all of the potential risks that we may face. For more information on how we manage risks, see Managing Risk in the MD&A beginning on page 44. For more information about the risks contained in this section,
@wesslen
wesslen / ibis-examples.ipynb
Last active March 12, 2024 17:24
ibis-examples.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wesslen
wesslen / duplicates.py
Created February 15, 2024 18:51
Check for duplicate inputs (text) for .jsonl files
import json
import logging
import typer
from pathlib import Path
from typing import List
from prodigy import set_hashes
logging.basicConfig(format='%(message)s', level=logging.INFO)
def process_file(file_path: Path, all_hashes: dict) -> dict:
# vs code create folder
# check python versions
python -v # make sure 3.9-3.12
pip -v
# create venv
python -m virtualenv venv
source venv/bin/activate
@wesslen
wesslen / create_data.py
Last active October 17, 2023 03:01
GPT3.5 fine tuning dummy example
import os
import json
import typer
app = typer.Typer()
def process_scripts(input_prompts_file: str, scripts_folder: str, output_file: str):
output_data = []
# Read input prompts from the .jsonl file