Skip to content

Instantly share code, notes, and snippets.

View tomplex's full-sized avatar

Tom Caruso tomplex

View GitHub Profile
@tomplex
tomplex / scrape.py
Created July 9, 2016 17:48
Convert jquery response to json
import requests
import json
def convert_to_json(url):
# get the response
r = requests.get(url)
# get the content of the response and decode from bytes to str in UTF-8
response = r.content.decode()
# the response is a jquery object, so we need to strip off the extra jquery nonsense.
# the beginning of the json object is the '[' so...
@tomplex
tomplex / mirror-url.py
Created November 30, 2016 01:22
flask app to mirror url
from flask import Flask
import requests
app = Flask(__name__)
@app.route('/mirror-url/<url>')
def mirror(url):
r = requests.get('http://' + url)
return r.content
@tomplex
tomplex / time_test.py
Created April 9, 2017 00:58
Test how long it takes to get x pages from investorhub
import requests
import time
import sys
def main():
base_url = 'http://investorshub.advfn.com/boards/read_msg.aspx?message_id={}'
times = []
program_start = time.time()
num_pages = int(sys.argv[1])
@tomplex
tomplex / multiproc_download.py
Created April 12, 2017 00:15
How to use Python multiprocessing to get and process web pages asyncronously
__author__ = 'tom caruso'
from multiprocessing import Pool
import requests
import time
import sys
base_url = 'http://investorshub.advfn.com/boards/read_msg.aspx?message_id={id}'
start_message = 130084355
CREATE TABLE users(user_id SERIAL PRIMARY KEY, user_name TEXT);
CREATE TABLE mentions(mention_id SERIAL PRIMARY KEY, mention_symbol TEXT, mention_user INT REFERENCES users(user_id), mention_date TIMESTAMP DEFAULT now());
INSERT INTO users (user_name) VALUES ('Tom Taylor'), ('Tom Caruso'), ('Other Rando User');
INSERT INTO mentions (mention_symbol, mention_user) VALUES ('AMD', 2);
INSERT INTO mentions (mention_symbol, mention_user) VALUES ('APPL', 1);
INSERT INTO mentions (mention_symbol, mention_user) VALUES ('BLPD', 1);
INSERT INTO mentions (mention_symbol, mention_user) VALUES ('GALT', 1);
@tomplex
tomplex / geometry.go
Created May 7, 2017 19:36
geometry interface & implementations
package main
import (
"fmt"
"math"
)
type geometry interface {
area() float64
perim() float64
@tomplex
tomplex / dict_attr.py
Last active May 15, 2017 13:26
A class for which attributes can be accessed like a `dict`.
class DictAttr:
def __init__(self):
self.attr1 = "Let's go"
self.attr2 = "Pens!"
def __getitem__(self, item):
if getattr(self, item, None):
return getattr(self, item)
else:
@tomplex
tomplex / find_open_port.sh
Created June 15, 2017 15:12
Find an open port on the host machine
#!/bin/bash
read LOWERPORT UPPERPORT < /proc/sys/net/ipv4/ip_local_port_range
while :
do
PORT="`shuf -i $LOWERPORT-$UPPERPORT -n 1`"
ss -lpn | grep -q ":$PORT " || break
done
echo $PORT
@tomplex
tomplex / clean-up-boot-partition-ubuntu.md
Created November 22, 2017 03:57 — forked from ipbastola/clean-up-boot-partition-ubuntu.md
Safest way to clean up boot partition - Ubuntu 14.04LTS-x64

Safest way to clean up boot partition - Ubuntu 14.04LTS-x64

Reference

Case I: if /boot is not 100% full and apt is working

1. Check the current kernel version

$ uname -r 
@tomplex
tomplex / container_stop_rm
Created December 2, 2017 01:42
Stop & remove Docker container with completions
#! /usr/bin/env bash
container_stop_rm () {
docker rm $(docker stop $1)
}
_container_stop_rm () {
containers=$(docker ps --format '{{ .Names }}')
COMPREPLY=( $(compgen -W "$containers" -- "${COMP_WORDS[COMP_CWORD]}" ) )