Skip to content

Instantly share code, notes, and snippets.

@nreimers
nreimers / cohere_rerank_v3_json.py
Last active June 12, 2024 07:57
Cohere Rerank V3 Example with JSON
# Make sure to have the newest Cohere SDK installed:
# pip install -U cohere
# Get your free API key from: www.cohere.com
import cohere
cohere_key = "<<YOUR_API_KEY>>"
co = cohere.Client(cohere_key)
# Lets define some JSON with our documents. Here we use a JSON to represent emails
# with different fields. In the call to co.rerank we can specify which
# This example shows how to use Cohere binary embeddings to get a 32x reduction in memory
# and up to a 40x faster search speed.
# You need the Cohere Python SDK as well as faiss
# pip install cohere faiss-cpu numpy
import faiss
import cohere
import numpy as np
@nreimers
nreimers / Opensearch_int8_search.py
Last active March 27, 2024 12:29
OpenSearch int8 search with Cohere
# This code shows how to index data using Cohere Embed v3 byte (int8) embeddings.
# This gives you a 4x memory reduction while keeping 99.9% of the search quality.
# Make sure to have OpenSearch running with at least version 2.9. E.g. by using docker:
# docker run -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:2.11.1
# You also need the OpenSearch python client installed.
# pip install cohere opensearch-py
from opensearchpy import OpenSearch, helpers
import cohere
This file has been truncated, but you can view the full file.
{"wiki_id": 3524766, "url": "https://en.wikipedia.org/wiki?curid=3524766", "views": 5409.5609619796405, "langs": 184, "title": "YouTube", "text": "YouTube is a global online video sharing and social media platform headquartered in San Bruno, California. It was launched on February 14, 2005, by Steve Chen, Chad Hurley, and Jawed Karim. It is owned by Google, and is the second most visited website, after Google Search. YouTube has more than 2.5 billion monthly users who collectively watch more than one billion hours of videos each day. , videos were being uploaded at a rate of more than 500 hours of content per minute.", "paragraph_id": 0, "id": 1}
{"wiki_id": 3524766, "url": "https://en.wikipedia.org/wiki?curid=3524766", "views": 5409.5609619796405, "langs": 184, "title": "YouTube", "text": "In October 2006, YouTube was bought by Google for $1.65\u00a0billion. Google's ownership of YouTube expanded the site's business model, expanding from generating revenue from advertisements alone, to offering paid content
@nreimers
nreimers / cohere_embed_v3.py
Last active April 8, 2024 19:26
Cohere Embed V3
# This snippet shows and example how to use the Cohere Embed V3 models for semantic search.
# Make sure to have the Cohere SDK in at least v4.30 install: pip install -U cohere
# Get your API key from: www.cohere.com
import cohere
import numpy as np
cohere_key = "{YOUR_COHERE_API_KEY}" #Get your API key from www.cohere.com
co = cohere.Client(cohere_key)
docs = ["The capital of France is Paris",
# 1) Install dependencies: pip install cohere
# 2) Get your Cohere API key and past it below
import cohere
# Get your cohere API key on: www.cohere.com
co = cohere.Client("<<YOUR_COHERE_API_KEY>>>")
# 1) Install dependencies: pip install cohere datasets elasticsearch==8.6.2
# 2) Start a local Elasticsearch server: docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:8.6.2
# 3) Get your Cohere API key and past it below
from elasticsearch import Elasticsearch, helpers
import cohere
from datasets import load_dataset
# Get your cohere API key on: www.cohere.com