Skip to content

Instantly share code, notes, and snippets.

View z-ai-lab's full-sized avatar
:octocat:

z z-ai-lab

:octocat:
View GitHub Profile
@z-ai-lab
z-ai-lab / neo_read_parquet.py
Created July 12, 2023 15:04
load parquet into neo4j
#!/Library/Frameworks/Python.framework/Versions/3.11/bin/python3
import pandas as pd
from neo4j import GraphDatabase
import yaml
import datetime
import sys
import gzip
from zipfile import ZipFile
from urllib.parse import urlparse
@z-ai-lab
z-ai-lab / clone_pg_table.py
Created July 21, 2023 14:20
clone pg table to another db
from dotenv import dotenv_values
from sqlalchemy import create_engine, Table
from sqlalchemy.engine import Connection
from sqlalchemy.schema import MetaData
# Load the environment variables from the .env file
cfg = dotenv_values(".env")
# Connect to the source database
engine_src = create_engine(cfg['src_database'])
@z-ai-lab
z-ai-lab / README.md
Created July 25, 2023 00:38
Script to change git history VERY easily

change_git_history

Script to change git history VERY easily

How to use

curl 'https://raw.githubusercontent.com/erwanvivien/change_git_history/master/change_history.sh' > change_history.sh
chmod +x change_history.sh

# NB_commits defaults to 5
@z-ai-lab
z-ai-lab / README.md
Created August 8, 2023 20:16
NeoSchema

A schema layer harmoniously brings together the best of the flexibility ("anything goes!") of graph databases
and the "law and order" aspect of relational databases!

@z-ai-lab
z-ai-lab / finetune_llama2.py
Created August 8, 2023 22:44
easy llama 2 fine-tuning script
# Based on younesbelkada/finetune_llama_v2.py
# Install the following libraries:
# pip install accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7 scipy
from dataclasses import dataclass, field
from typing import Optional
import torch
from datasets import load_dataset
from transformers import (
@z-ai-lab
z-ai-lab / main.py
Created August 8, 2023 23:10
QA Chatbot streaming with source documents example using FastAPI, LangChain Expression Language, OpenAI, and Chroma.
"""QA Chatbot streaming using FastAPI, LangChain Expression Language , OpenAI, and Chroma.
Features
--------
- Persistent Chat Memory:
Stores chat history in a local file.
- Persistent Vector Store:
Stores document embeddings in a local vector store.
- Standalone Question Generation:
Rephrases follow-up questions to standalone questions in their original language.
@z-ai-lab
z-ai-lab / 00-fine-tuning.ipynb
Created August 29, 2023 17:06
gpt-3.5-agent-training
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@z-ai-lab
z-ai-lab / WorksheetDictWriter.py
Created September 5, 2023 14:33
A DictWriter interface for Google Spreadsheets
class WorksheetDictWriter:
"""
Something like a `csv.DictWriter`, except for a `gspread.Worksheet`
"""
def __init__(self, worksheet: gspread.Worksheet, fieldnames: list[str], column="A"):
self.fieldnames = fieldnames
self.worksheet = worksheet
self.column = column
self.current_row = 1
@z-ai-lab
z-ai-lab / finetune_llama_v2.py
Created September 5, 2023 19:50
Fine tune Llama v2 models on Guanaco Dataset
# coding=utf-8
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@z-ai-lab
z-ai-lab / mistral_haystack.ipynb
Created October 9, 2023 02:07
Mistral + Haystack: build RAG pipelines that rock 🤘
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.