fsndzomga/fetch-and-split.py

Created April 20, 2024 12:56

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/fsndzomga/f46ed3e01303bd41d19ed4ef0ae73c7b.js"></script>
Save fsndzomga/f46ed3e01303bd41d19ed4ef0ae73c7b to your computer and use it in GitHub Desktop.

Download ZIP

Raw

fetch-and-split.py

	docs = WebBaseLoader(url).load()
	text_splitter = RecursiveCharacterTextSplitter()
	documents = text_splitter.split_documents(docs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment