Skip to content

Instantly share code, notes, and snippets.

Keybase proof

I hereby claim:

  • I am jimr on github.
  • I am jimr (https://keybase.io/jimr) on keybase.
  • I have a public key whose fingerprint is 788D 1F2D A813 72EA 42A5 350A C19F 559A 4045 40D7

To claim this, I am signing this object:

@jimr
jimr / db_size_check.sql
Created September 16, 2015 13:52
Show the top 25 MySQL database tables by size on disk
SELECT
CONCAT(table_schema, '.', table_name) as 'table',
CONCAT(ROUND(table_rows / 1000000, 2), 'M') rows,
CONCAT(ROUND(data_length / (1024 * 1024 * 1024), 2), 'G') data,
CONCAT(ROUND(index_length / (1024 * 1024 * 1024), 2), 'G') idx,
CONCAT(ROUND((data_length + index_length) / (1024 * 1024 * 1024), 2), 'G') total_size,
ROUND(index_length / data_length, 2) idxfrac
FROM information_schema.TABLES
-- WHERE table_schema = 'some_db'
ORDER BY data_length + index_length DESC
@jimr
jimr / pagination.py
Created December 9, 2014 14:56
flickr.py search pagination usage example
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import flickr
flickr.API_KEY = '<your API key>'
flickr.API_SECRET = '<your API secret>'
page = 1
per_page = 10
@jimr
jimr / locations.py
Last active August 29, 2015 14:11
Example flickr.py usage for getting user locations
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import flickr
flickr.API_KEY = '<your API key>'
flickr.API_SECRET = '<your API secret>'
# get some photos
photos = flickr.photos_search(tags='manhattan', page='1', per_page='10')
@jimr
jimr / email_finder.py
Created July 21, 2014 08:40
Good Enough(TM) email address finder
import re
pattern = re.compile(
r'(mailto:)?(?P<email>[^(\s|<|\[))]*@[^\s]*\.[^(\s|>|;|,|\])]*)'
)
test_data = '''
This is a@b.com string with some Email Addresses
<email@addresses.com> in it. There are also
some mailto:test@example.com addresses in here too.
@jimr
jimr / coord_cleaner.py
Last active August 29, 2015 14:04
Helper function for cleaning up user input geographic coordinates. Call get_coords(crappy_x, crappy_y) to get cleaned coordinates. No guarantees, check your results, etc.
def clean_point(point):
# junk
point = point.strip()
junk = [u'\ufeff', ' ', u'\xa0', u'\u2013'] # [, ,  , –]
for symbol in junk:
point = point.replace(symbol, '')
point = point.lstrip('+')
point = point.lstrip("'") # leading quote chars to fool excel
point = point.lstrip(',')
@jimr
jimr / find_a_ride.py
Last active August 29, 2015 14:03
Get NextBike rental station numbers for Bath. I'm guessing about the time zone data for when the feed was updated (no docs that I can find).
import datetime
import sys
import urllib2
import xml.etree.ElementTree as ET
url = "https://nextbike.net/maps/nextbike-live.xml"
stations = sys.argv[1:]
xml = urllib2.urlopen(url).read()
root = ET.fromstring(xml)
@jimr
jimr / fake_request.py
Last active July 4, 2022 09:40
Generate fake WSGIRequest objects for use in Django views
def fake_request(method=None, fake_user=False):
'''Returns a fake `WSGIRequest` object that can be passed to viewss.
If `fake_user` is `True`, we attach a random staff member to the request.
Even if not set, you can still do this manually by setting the `user`
attribute on the returned object.
The `GET` and `POST` `QueryDict` objects are mutable::
req = fake_request(mutable=True)
@jimr
jimr / count_the_hours.py
Created December 6, 2013 11:38
Tally up records from httpd access logs by hour of day. Usage: python count_the_hours.py /path/to/access.log.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import itertools
import re
import sys
from datetime import datetime as dt
pattern = re.compile(r'.* \[([^]]*) \+[0-9]+\] .*')
@jimr
jimr / shape_merge.py
Created August 15, 2013 14:35
Merge zipped shapefiles containing identical (or very similar) attribute tables.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Requires pyshp: https://pypi.python.org/pypi/pyshp
#
import os
import shapefile
import shutil
import tempfile