Skip to content

Instantly share code, notes, and snippets.

@feuyeux
Last active February 22, 2024 15:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save feuyeux/0bebd7c0143953aaa809052c567b8ce0 to your computer and use it in GitHub Desktop.
Save feuyeux/0bebd7c0143953aaa809052c567b8ce0 to your computer and use it in GitHub Desktop.
RAG sample using Langchain and Ray

https://github.com/ray-project/langchain-ray这个仓库展示了3个ray和langchain结合的示例,包括源代码、视频讲解、博客文档,对快速认知这两个工具及如何结合使用,非常有帮助。(但不知道出于什么原因,仅不到一年,这个仓库的状态已经是Public archive了。)

让我很有印象的是RAG应用示例。

首先是特征向量化过程(Embedding/Vector Store)。Ray Data提供了并行处理数据能力,将原始PDF文件向量化后存储/索引到FAISS。这个过程如果单纯使用langchain实现,这个过程是小时级的;而引入ray data后这个过程是分钟级的。

embedding-why-do-I-need-to-parallelize-this

embedded-stages-of-our-pipeline

单纯使用langchain进行向量存储的示例代码:https://github.com/ray-project/langchain-ray/blob/main/embedding_pdf_documents/embedding_serial.py

langchain+ray进行向量存储的示例代码:https://github.com/ray-project/langchain-ray/blob/main/embedding_pdf_documents/embedding_ray.py

有了向量存储,就可以实现Retrieval QA了,通过Ray Serve部署Question Answering Service:

qna-save-search-querie

在此基础上,将问答结果作为Prompt模板的参数输入,生成最终的Prompt,喂给LLM,生成最终响应:

qna-answering-questions-with

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment