Skip to content

Instantly share code, notes, and snippets.

@CristianContrera95
CristianContrera95 / Dockerfile
Last active June 24, 2021 23:53
Custom images for Data bricks
FROM databricksruntime/python:latest
# RUN apt-get install curl
# copiamos el archivo con las librerias a nuestro docker
COPY requirements.txt ./
# instalamos todas las librerias de python que se indican en requirements
RUN /databricks/conda/envs/dcs-minimal/bin/pip install -r requirements.txt
@CristianContrera95
CristianContrera95 / thread.py
Last active July 14, 2021 17:32
Thread-based parallelism
# import Thread from standard python library
from threading import Thread
# define function to be executed on the thread
def upload_files(files):
""" Upload given files to some storage """
url = 'https://own-storage.net/upload'
for filename in filenames:
with open(filename, 'rb') as file:
import tempfile
from pathlib import Path
# Define users list as data example
users = ['Micho', 'Tito', 'Gordo', 'Cabezon']
# create a new (write/read mode) temporary file that will be delete when file close
tmp = tempfile.NamedTemporaryFile(mode='w+')
# Save users into tmp file
@CristianContrera95
CristianContrera95 / Ngrok.md
Last active April 1, 2021 16:51
Share local server to internet and other collaborators

Share API with ngrok

Si trabajamos usando APIs-REST, Dashboards Web, Documentacion (o algo con un servidor) muchas veces necesitamos compartir nuestro servidor local con otras partes de nuestro equipo, vamos a mostrar como Ngrok para esto en linux (ubuntu 20.04):

Instalacion

snap install ngrok

Uso

Corremos el servidor (en este caso podria ser una api de python), y luego le indicamos a ngrok sobre que puerto compartir nuestra red local

@CristianContrera95
CristianContrera95 / parallel.py
Last active March 23, 2021 16:01
Parallel ejecution with Python
from multiprocessing import Pool, cpu_count
# Define functions to execute on new process
def get_average(array):
"""Return average for array"""
assert len(array) != 0, "argument 'array' must have at least 1 item"
return sum(array) / len(array)
@CristianContrera95
CristianContrera95 / requests.py
Created March 18, 2021 01:25
Make requests to API-REST with Python
# install this librarie with: pip install requests
import requests
# define host where API-REST is running
host_url = "https://swapi.dev/api/"
# make a GET requests to receive info
response = requests.get(host_url+"people/1/")
if response.status_code == 200:
# if requested was executed successfully print response on json format
@CristianContrera95
CristianContrera95 / command.py
Created March 18, 2021 00:45
Run console command with Python
import os
import sys
from subprocess import Popen
# Perform a call to clone yolov3 git repository
Popen(args=["git", "clone", "https://github.com/eriklindernoren/PyTorch-YOLOv3.git"],
cwd=os.getcwd(),
stdin=subprocess.PIPE,
stdout=sys.stdout, # use sys.stdout and sys.stderr to get in real time logs from command
stderr=sys.stderr
@CristianContrera95
CristianContrera95 / helpers.sh
Created March 18, 2021 00:35
Load bash functions from another script
#!/bin/bash
# In this file we define some functions to be user for other scripts
active_env () {
# activate enviroment shell
pipenv shell
}
generate_requirements() {
# generate requirements files
@CristianContrera95
CristianContrera95 / read_password.sh
Last active March 18, 2021 00:24
Hidden password in bash script
#!/bin/bash
echo "Ingrese su usuario"
read USER
echo "Ingrese la contraseña para $USER"
# With the -s flag the input text will be hidden
read -s PASSWORD
# this is only use example (docker login)
docker login -u $USER -p $PASSWORD
@CristianContrera95
CristianContrera95 / mongodb.py
Last active March 18, 2021 16:34
Connect to MongoDB with Python
# install pymongo with: pip install pymongo
import pymongo
# mongodb connection data
MONGO_DB {
'MONGO_SERVER': 'mongodb://127.0.0.1:27017/',
'DATABASE_NAME': 'mi_database',
'MONGO_USER': 'user',
'MONGO_PWD': '1234'
}