Skip to content

Instantly share code, notes, and snippets.

View alonsoir's full-sized avatar
👋
Deep learning with python time!

@alonso_isidoro alonsoir

👋
Deep learning with python time!
84 followers · 23 following
View GitHub Profile
@alonsoir
alonsoir / main-spark-python-generator.py
Created April 4, 2024 10:16
I can connect to a spark cluster to ask it to do some task using phrases. This is something really interesting.
┌<▸> ~/D/spark-3.4.2-bin-hadoop3
└➤
./sbin/start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.4.0
starting org.apache.spark.sql.connect.service.SparkConnectServer, logging to /Users/aironman/Downloads/spark-3.4.2-bin-hadoop3/logs/spark-aironman-org.apache.spark.sql.connect.service.SparkConnectServer-1-MacBook-Pro-de-Alonso.local.out
┌<▸> ~/D/spark-3.4.2-bin-hadoop3
└➤
lsof -iTCP -sTCP:LISTEN -n -P | grep 15002
java 4566 aironman 445u IPv6 0x75d482d23c96e711 0t0 TCP *:15002 (LISTEN)
@alonsoir
alonsoir / Pipfile
Created April 2, 2024 12:03
A sample about how can i ask to openAI to generate python code using some instructions and a special agent. Look out the second answer, i wasnt sure the necessary packages for PyTorch, i asked it in spanish, and solves the question.
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
langchain = "*"
python-dotenv = "*"
black = "*"
langchain-experimental = "*"
@alonsoir
alonsoir / README.md
Last active March 28, 2024 11:57
langchain with Pinecone and a index loaded with your own dataset.
@alonsoir
alonsoir / download_docs.py
Created March 27, 2024 11:01
scrapping with BeautifulSoup. Probably it will not work if webserver is well configured. The second at least will download the html files, but is useless if content is not downloaded.
import requests
from bs4 import BeautifulSoup
import os
import urllib
import time
# Inicio del temporizador
start_time = time.time()
# Ejecución de la tarea
@alonsoir
alonsoir / checkVirusTotal.py
Created March 26, 2024 12:53
to check, theorically it is able to talk with VT api to resolve health files.
import os
import time
import requests
from dotenv import load_dotenv # Assuming you have `python-dotenv` installed
def is_file_safe(file_path, api_key):
"""
Checks a file's safety using VirusTotal API.
@alonsoir
alonsoir / Acerca de FAISS.txt
Last active March 26, 2024 16:55
Working with FAISS instead of Pinecone. It is like a local VectorDB, it applies cosine similarity too, perfect for local development.
Aprovechar al máximo FAISS: preguntas y obtención de fuentes
Limitaciones de FAISS con textos técnicos:
FAISS es una biblioteca poderosa para la búsqueda y recuperación de información, pero tiene limitaciones al trabajar con textos técnicos como el de tu PDF de mecánica cuántica. Estos textos suelen tener un lenguaje complejo, especializado y con tecnicismos que la biblioteca no comprende del todo.
Preguntas que puedes hacer a FAISS:
Preguntas simples: Son ideales para obtener información general o superficial del contenido.
Preguntas sobre entidades: Puedes preguntar por nombres de personas, lugares, organizaciones o conceptos específicos mencionados en el texto.
@alonsoir
alonsoir / main.py
Created March 25, 2024 12:36
Exploring more options with langchain and VectorDBQA. Usando Pinecone como base de datos vectorial. VectorDBQA es un componente de Langchain que se utiliza específicamente para responder preguntas a partir de una base de datos vectorial. Funciones de VectorDBQA: Interfaz: VectorDBQA facilita la interacción con la base de datos vectorial Pinecone…
import os
from langchain.chains.retrieval_qa.base import VectorDBQA
from langchain_community.document_loaders import TextLoader
from langchain_openai import OpenAIEmbeddings, OpenAI
from langchain_pinecone import PineconeVectorStore
from langchain_text_splitters import CharacterTextSplitter
import logging
logger = logging.getLogger(__name__)
@alonsoir
alonsoir / Pipfile
Created March 21, 2024 11:50
introduction to vectorDB like Pinecone with langchain and openai
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
langchain = "*"
black = "*"
pinecone-client = "*"
openai = "*"
@alonsoir
alonsoir / Pipfile
Last active March 20, 2024 11:03
how to use custom tools with templates from langchain-hub
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
langchain = "*"
openai = "*"
black = "*"
python-dotenv = "*"
@alonsoir
alonsoir / output.txt
Created March 19, 2024 13:12
Un ejemplo de un agente langchain que sigue un patrón template.
/Users/aironman/.local/share/virtualenvs/react-langchain-TwV6jvy5/bin/python /Users/aironman/Desktop/react-langchain/main.py
/Users/aironman/.local/share/virtualenvs/react-langchain-TwV6jvy5/lib/python3.12/site-packages/langchain/chat_models/__init__.py:31: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:
`from langchain_community.chat_models import ChatOpenAI`.
To install langchain-community run `pip install -U langchain-community`.
warnings.warn(
Hello ReAct LangChain!, Alonso
/Users/aironman/.local/share/virtualenvs/react-langchain-TwV6jvy5/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.chat_models.openai.ChatOpenAI` was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and sh