Skip to content

Instantly share code, notes, and snippets.

View tazarov's full-sized avatar
🦖
Tell me and I forget. Teach me and I remember. Involve me and I learn.

Trayan Azarov tazarov

🦖
Tell me and I forget. Teach me and I remember. Involve me and I learn.
View GitHub Profile
[Unit]
Description = Chroma Service
After = network.target
[Service]
Type = simple
User = root
Group = root
WorkingDirectory = /chroma
ExecStart=/usr/local/bin/chroma run --host 127.0.0.1 --port 8000 --path /chroma/data --log-path /var/log/chroma.log
[Unit]
Description = Chroma Docker Service
After = network.target docker.service
Requires = docker.service
[Service]
Type = forking
User = root
Group = root
WorkingDirectory = /home/admin/chroma
@tazarov
tazarov / wal_clean.py
Last active January 18, 2024 12:27
Clean ChromaDB WAL (⚠️ This script has been superseded by https://github.com/amikos-tech/chromadb-ops ⚠️)
#!/usr/bin/env python3
# Call the script: python wal_clean.py ./chroma-test-compact
#!/usr/bin/env python3
# Call the script: python wal_clean.py ./chroma-test-compact
import argparse
import importlib
import os
import sqlite3
import typer
@tazarov
tazarov / cleanup.sh
Created January 9, 2024 13:56
Clean up ChromaDB defunct binary indices.
#!/bin/bash
if ! command -v sqlite3 &> /dev/null; then
echo "sqlite3 could not be found. Please install sqlite3 and try again."
exit 1
fi
PERSIST_DIR=$(realpath "$1")
BACKUP_DIR=$(realpath "$2")
@tazarov
tazarov / docker-compose.yaml
Last active December 31, 2023 15:29
Minimal Docker Compose for Chroma
version: '3.9'
networks:
net:
driver: bridge
services:
chromadb:
image: chromadb/chroma:latest
volumes:
- ./chromadb:/chroma/chroma
@tazarov
tazarov / llama_embeddings_for_chroma.py
Last active December 13, 2023 13:19
Chroma and LlamaIndex both offer embedding functions which are wrappers on top of popular embedding models. Unfortunately Chroma and LI's embedding functions are not compatible with each other. Below we offer an adapters to convert LI embedding function to Chroma one.
from llama_index.embeddings import OpenAIEmbedding
from llama_index.embeddings.base import BaseEmbedding
import chromadb
from chromadb.api.types import EmbeddingFunction
class LlamaIndexEmbeddingAdapter(EmbeddingFunction):
def __init__(self,ef:BaseEmbedding):
self.ef = ef
def __call__(self, input: Documents) -> Embeddings:
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@tazarov
tazarov / recent_entries.ipynb
Created November 29, 2023 09:20
Query collection with date filter
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@tazarov
tazarov / keyword_search.py
Created November 24, 2023 12:55
Keyword Search in Chroma
import chromadb
from chromadb.config import Settings
client = chromadb.PersistentClient(path="test", settings=Settings(allow_reset=True))
client.reset()
col = client.get_or_create_collection("test")
col.add(ids=["1", "2", "3"], documents=["He is a technology freak and he loves AI topics", "AI technology are advancing at a fast pace", "Innovation in LLMs is a hot topic"])
col.query(query_texts=["technology"], where_document={"$or":[{"$contains":"technology"}, {"$contains":"freak"}]})
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.