Skip to content

Instantly share code, notes, and snippets.

@rajtilakjee
Created April 9, 2023 12:55
Show Gist options
  • Save rajtilakjee/7020232f2d7f931a7244e3f2b0dba90d to your computer and use it in GitHub Desktop.
Save rajtilakjee/7020232f2d7f931a7244e3f2b0dba90d to your computer and use it in GitHub Desktop.
Using LangChain with ChatGPT to create a web scraper
from langchain.agents.agent_toolkits import create_python_agent
from langchain.tools.python.tool import PythonREPLTool
from langchain.python import PythonREPL
from langchain.llms.openai import OpenAI
import os
os.ENVIRON['OPENAI_API_KEY'] = "sk-xxxxxxxxxxxxxxx"
agent_executor = create_python_agent(
llm=OpenAI(temperature=0, max_tokens=1000),
tool=PythonREPLTool(),
verbose=True
)
agent_executor.run("""scrape the homepage of microsoft.com using BeautifulSoup""")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment