Skip to content

Instantly share code, notes, and snippets.

@davidread
davidread / README.md
Created June 4, 2014 14:49
data.gov.uk broken links 4/6/14

README is empty

@davidread
davidread / gist:b318ed3c6604384b7f7e
Last active August 29, 2015 14:04
These are the data.gov.uk themes. The INSPIRE themes are mapped to them, according to the "gemet" sections. e.g. under the DGU Theme "Mapping" you see INSPIRE theme "Addresses". The descriptions of the 34 INSPIRE themes are here: http://inspire.ec.europa.eu/index.cfm/pageid/2/list/7
[ {"title": "Crime & Justice",
"description": "Statistics on crimes, policing, justice, courts, prisons, offenders, borders, and immigration.",
},
{"title": "Defence",
"description": "Statistics on the armed forces, Ministry of Defence, workforce, health & safety and search & rescue.",
},
{"title": "Economy & Business",
"description": "Industry and small businesses. Statistics, surveys, economic indicators, import/exports and trade. Excludes government spending transactions & salaries. Excludes household finances",
},
{"title": "Education",
@davidread
davidread / gist:11fafa9d5716766dd053
Created October 2, 2014 16:15
DGU To Go vagrant output
dread@dmbp:~/v-to-go$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'precise64'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
@davidread
davidread / gist:979a7d63f6d813052255
Created November 3, 2014 13:34
Metadata provenance
Example dataset extra field "metadata_provenance"
This dataset originates at Barnet Open Data, is then harvested to London Datastore and then harvested onto data.gov.uk. When it is viewed in data.gov.uk it has this metadata_provenance:
[
{
"activity_occurred": "2014-10-21T09:04:19.753433",
"activity": "harvest",
"harvest_source_url": "https://open.barnet.gov.uk/",
"harvest_source_title": "Barnet Open Data",
<a href="fdfd"></a>
import nose.tools
import ckan.model as model
assert_equals = nose.tools.assert_equals
assert_not_equals = nose.tools.assert_not_equals
Resource = model.Resource
class TestResource(object):
@davidread
davidread / pgbouncer.ini
Created March 19, 2015 14:02
data.gov.uk pgbouncer config /etc/pgbouncer/pgbouncer.ini
;; database name = connect string
[databases]
ckan = host=127.0.0.1 dbname=ckan user=dgu password=REDACTED pool_size=40
;;dgudatastore = host=127.0.0.1 dbname=dgudatastore user=dgu password=REDACTED pool_size=10
;;dgudatastore_ro = host=127.0.0.1 dbname=dgudatastore user=dguro password=REDACTED pool_size=10
dgucelery = host=127.0.0.1 dbname=dgucelery user=dgu password=redacted pool_size=5
;; Configuation section
;; Configuation section
[pgbouncer]
dread@dmbp:~$ git clone git@github.com:datagovuk/dgu-vagrant-puppet v-test
Cloning into 'v-test'...
remote: Counting objects: 1742, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 1742 (delta 2), reused 0 (delta 0), pack-reused 1733
Receiving objects: 100% (1742/1742), 40.88 MiB | 2.02 MiB/s, done.
Resolving deltas: 100% (836/836), done.
Checking connectivity... done.
dread@dmbp:~$ cd v-test
dread@dmbp:~/v-test$ git checkout togo
>>> requests.get('http://uk-air.defra.gov.uk/datastore/pcb/2010_TOMPs_PCB_Data.xlsx', headers={'User-agent': 'python-requests'})
<Response [403]>
>>> requests.get('http://uk-air.defra.gov.uk/datastore/pcb/2010_TOMPs_PCB_Data.xlsx', headers={'User-agent': 'python-request'})
<Response [200]>
diff --git a/ckan/tests/functional/test_user.py b/ckan/tests/functional/test_user.py
index 4ae63c0..3afc1f2 100644
--- a/ckan/tests/functional/test_user.py
+++ b/ckan/tests/functional/test_user.py
@@ -168,12 +168,14 @@ class TestUserController(FunctionalTestCase, HtmlCheckMethods, PylonsTestCase, S
# first get redirected to user/logged_in
assert_equal(res.status, 302)
- assert res.header('Location').startswith('http://localhost/user/logged_in')
+ assert res.header('Location').startswith('http://localhost/user/logged_in') or \