Skip to content

Instantly share code, notes, and snippets.

<!doctype html>
<html>
<head>
<meta charset="UTF-8">
<title>Document</title>
</head>
<body>
<div>
<form>
<!doctype html>
<html>
<head>
<meta charset="UTF-8">
<title>Document</title>
</head>
<body>
<div>
<form>
#! /bin/bash
### IMPORTANT ###
# Remove all 3 instances of /path/to/ and replaces them with proper paths
#################
logfile='/path/to/logs.txt'
start_text='Starting MGS_scraper.py'
current_time=$(date)

Improve Python Scraper Uptime

Start MGS_scraper.py on boot

  1. $ crontab -e
  2. Add @reboot sh /path/to/find_or_start_scraper.sh
  3. Be sure to replace /path/to/ with the correct path.

Restart the scraper every X minutes, if it has crashed

from bs4 import BeautifulSoup
import csv
from datetime import datetime
import html
import requests
import sqlite3
from time import time, sleep
# README
# pip install beautifulsoup4
@andygimma
andygimma / flatten.js
Created September 27, 2019 15:42
Flatten array
describe("flatten an array of arbitrarily nested arrays of integers into a flat array of integers", function () {
it("Should return empty array when given empty array provided", function () {
expect(flatten([])).toEqual([]);
});
it("Should return a flat array when given a flat array provided", function () {
expect(flatten([1])).toEqual([1]);
});
function convertDecimalsToDegrees(num) {
var integer = num.toString().split('.')[0]
var decimal = '.' + num.toString().split('.')[1]
var convert = decimal * 60;
var min = convert.toString()
return integer + ' ' + min;
}

Keybase proof

I hereby claim:

  • I am andygimma on github.
  • I am andygimma (https://keybase.io/andygimma) on keybase.
  • I have a public key ASAlRW9l5JgmaW0lzmssQza-kiic8E0Qfm8VP4uAjTaC9wo

To claim this, I am signing this object:

#!/usr/bin/env python
import urllib2
import thread
import sys
def fetch_api(url):
res = urllib2.urlopen(url)
def fetch_api_hits(url, number_of_hits):
for i in range(number_of_hits):
var sites = {
A1: {
name: "Wendy Williams",
city: "Houston",
work_type: "flood",
urgency: 5
},
A2: {
name: "Jello Biafra",
city: "Houston",