Skip to content

Instantly share code, notes, and snippets.

@rossjones
rossjones / gist:1603179
Created January 12, 2012 21:17
Serve file from folder for flask
# Pass filename to send from the static subdirectory
directory = os.path.join( os.path.dirname(os.path.realpath(__file__)), 'static' )
return send_from_directory(directory, filename)
def image_path(self):
"""Returns the path to the images
"""
from django.conf import settings
if hasattr(settings,'RANDOM_IMAGE_REGEX'):
for x in settings.RANDOM_IMAGE_REGEX:
p,g = x
if g and p in self.path:
return "%s%s/%s" % (settings.MEDIA_ROOT, self.path, g)
@rossjones
rossjones / gist:2936408
Created June 15, 2012 13:11
random parsing heuristics
Read the file a line at a time, and if we find a line that begins VEP
then we're in business.
1. Store VEPXXXXXX somewhere, until we get a whitespace.
2. Rest of line (strip off whitespace) is a name.
3. Keep reading lines until the end of a line contains 2 chars a space
and 9 digits (MO 631464129). Until that point, each row is part of
the address.
4. The next line will start with a product name, and the next line will
Failed: 500 code: {"error":"SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[noDF3TLpSC6uLoDruk2iwg][ckan-demo.ckan.org][4]: RemoteTransportException[[Bloodwraith][inet[/193.34.146.144:9300]][search/phase/query]]; nested: QueryPhaseExecutionException[[ckan-demo.ckan.org][4]: query[filtered(ConstantScore(ConstantScore(*:*)))->FilterCacheFilterWrapper(_type:281dffa6-ea9b-4446-be41-05dced06591f)],from[0],size[100],sort[<custom:\"Date\": org.elasticsearch.index.field.data.strings.StringFieldDataType$1@777c1ce3>]: Query Failed [Failed to execute main query]]; nested: IOException[Can't sort on string types with more than one value per doc, or more than one token per field]; }{[noDF3TLpSC6uLoDruk2iwg][ckan-demo.ckan.org][3]: RemoteTransportException[[Bloodwraith][inet[/193.34.146.144:9300]][search/phase/query]]; nested: QueryPhaseExecutionException[[ckan-demo.ckan.org][3]: query[filtered(ConstantScore(ConstantScore(*:*)))->FilterCacheFilterWrapper(_type:281dffa6-ea9b-4446
Failed: 500 code:
{"error":"SearchPhaseExecutionException
[Failed to execute phase [query], total failure;
shardFailures {[noDF3TLpSC6uLoDruk2iwg][ckan-demo.ckan.org][4]:
RemoteTransportException[[Bloodwraith][inet[/193.34.146.144:9300]][search/phase/query]]; nested: QueryPhaseExecutionException[[ckan-demo.ckan.org][4]: query[filtered(ConstantScore(ConstantScore(*:*)))->FilterCacheFilterWrapper(_type:281dffa6-ea9b-4446-be41-05dced06591f)],from[0],size[100],sort[<custom:\"Date\": org.elasticsearch.index.field.data.strings.StringFieldDataType$1@777c1ce3>]: Query Failed [Failed to execute main query]]; nested: IOException[Can't sort on string types with more than one value per doc, or more than one token per field]; }{[noDF3TLpSC6uLoDruk2iwg][ckan-demo.ckan.org][3]: RemoteTransportException[[Bloodwraith][inet[/193.34.146.144:9300]][search/phase/query]]; nested: QueryPhaseExecutionException[[ckan-demo.ckan.org][3]: query[filtered(ConstantScore(ConstantScore(*:*)))->FilterCacheFilterWrapper(_type:281dffa6-ea9b
Traceback (most recent call last):
File "/Users/ross/Work/Servercode/okfn/pyenv/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.1.2', 'console_scripts', 'nosetests')()
File "/Users/ross/Work/Servercode/okfn/pyenv/lib/python2.7/site-packages/nose/core.py", line 118, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Users/ross/Work/Servercode/okfn/pyenv/lib/python2.7/site-packages/nose/core.py", line 135, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Users/ross/Work/Servercode/okfn/pyenv/lib/python2.7/site-packages/nose/config.py", line 339, in configure
@rossjones
rossjones / gist:3749344
Created September 19, 2012 12:16
Random code
import csv
from datetime import datetime
from subprocess import Popen, PIPE
from models import (Consignee, ConsigneeItem)
TABLES = [
("Consignee", Consignee,),
("Item", ConsigneeItem,),
]
@rossjones
rossjones / datafart example
Created December 22, 2012 20:27
Grabs a csv from scraperwiki, pulls a couple of fields and then feeds it (minus the CSV header) to datafart.
curl -s "https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=tourist_arrival_statistics_sikkim&query=select+*+from+swdata&apikey=" | awk -F"," '{print $2, $3}' | tail -n+2 | curl -s --data-binary @- datafart.com
@rossjones
rossjones / datafart.py
Created December 22, 2012 21:00
Thrown together script to send a URL and col positions to datafart
#!/usr/bin/env python
import csv
import sys
import urllib2
import webbrowser
def send_to_datafart(url, cols, skip_headers):
handle = urllib2.urlopen(url)
reader = csv.reader(handle)
if skip_headers:
@rossjones
rossjones / gist:4647595
Last active October 3, 2018 16:10
google spreadsheet to CSV
<?php
function get_details($key, $email) {
$csv = 'https://spreadsheets.google.com/pub?key=' . $key . '&output=csv&ndplr=1';
$session = curl_init();
curl_setopt( $session, CURLOPT_HEADER, 0 );
curl_setopt( $session, CURLOPT_RETURNTRANSFER, TRUE );
curl_setopt( $session, CURLOPT_URL, $csv );