Skip to content

Instantly share code, notes, and snippets.

@rkreddyp

rkreddyp/gist:4d63fa11ad95790be9581c60a1065d9b

Created October 27, 2023 01:18
Show Gist options
  • Save rkreddyp/4d63fa11ad95790be9581c60a1065d9b to your computer and use it in GitHub Desktop.
Save rkreddyp/4d63fa11ad95790be9581c60a1065d9b to your computer and use it in GitHub Desktop.
Download ZIP
pinecone create
Raw
gistfile1.txt
from langchain.document_loaders import WebBaseLoader
from langchain.document_loaders import TextLoader
url = 'https://developer.okta.com/docs/reference/api/event-types/#catalog'
loader = TextLoader ('/tmp/okta_events.txt')
scrape_data = loader.load()
index_name = 'oktaevents'
# Chunk your data up into smaller documents
text_splitter = RecursiveCharacterTextSplitter()
texts = text_splitter.split_documents(scrape_data)
Pinecone.from_documents(texts, embeddings, index_name=index_name)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment