Skip to content

Instantly share code, notes, and snippets.

View selenamarie's full-sized avatar

Selena Deckelmann selenamarie

View GitHub Profile
@selenamarie
selenamarie / gist:7208066
Created October 29, 2013 02:05
finding busted json
for i in `seq 1 59`; do
num=$i
num2=$(($i+1))
psql breakpad -c "SELECT 1 from raw_crashes where json_object_field_text(raw_crash, 'AdapterVendorID') is not NULL and json_object_field_text(raw_crash, 'AdapterDeviceID') is NOT NULL and date_processed >= '2013-10-26 17:09:${num}+00'::timestamptz AND date_processed <= '2013-10-26 17:09:${num2}+00'::timestamptz"
echo $i
done
selena@zhongzong:chesnok #522 12:15 :) hexo migrate wordpress ~/Downloads/selenadeckelmann.wordpress.2013-10-19.xml
Fetching /Users/selena/Downloads/selenadeckelmann.wordpress.2013-10-19.xml.
Parsing XML.
Analyzing.
1716 posts migrated.
import pika
parameters = pika.URLParameters('STUFF HERE')
connection = pika.BlockingConnection(parameters)
channel = connection.channel()
channel.queue_declare(queue='socorro.priority', durable=True)
with open('/tmp/uuid_list1000', 'r') as file:
for uuid in file.read().splitlines():
channel.basic_publish(exchange='', routing_key="socorro.priority", body=uuid, properties=pika.BasicProperties(delivery_mode=2))
print 'done with', uuid
connection.close()
Monitor, you were
a single point of failure
i will not miss, much.
@selenamarie
selenamarie / foss_skills.txt
Created July 12, 2013 20:35
This is a list of skills & concepts taught in "first patch", "beginner" and "introductory" free/open source software training programs I've been part of or read about. Anything I'm missing?
"Technical" skills:
* Distributed revision control
* Pair programming
* Testing
* Documentation
* Automation (for build, testing and deployment)
"Non-technical" skills:
* Finding documentation/help
* Mentoring
with exploitability_count AS (
select count(*) from exploitability_reports
WHERE
report_date BETWEEN %(start_date)s AND %(end_date)s
)
SELECT
exploitability_count.count
signature,

Title: Schema liberation with JSON and PLV8

JSON is the lingua franca of the web. Postgres supports JSON natively and the PLV8 Javascript engine as an extension. Full Javascript is supported inside the database, including direct access to tables and important database features.

And, developer-friendly features make it incredibly easy to transform your existing, normalized schemas into liberated JSON ones!

Tour of what putting a document store inside of Postgres looks like, including a look at a production use-case from Mozilla. crash-stats.mozilla.com is backed by a 2 TB Postgres database cluster that's adding about about 5 GB of JSON per week.

@selenamarie
selenamarie / json_example.sql
Created May 24, 2013 23:17
Playing around with JSON and json_enhancements
You are now connected to database "postgres" as user "postgres".
postgres:5432# create table birds (bird_info json);
CREATE TABLE
Time: 162.227 ms
postgres:5432# insert into birds VALUES( '{"name": "scrubjay", "where": "my backyard", "when": "2013-05-24"}');
INSERT 0 1
Time: 0.616 ms
postgres:5432# select * from birds;
bird_info
--------------------------------------------------------------------
@selenamarie
selenamarie / liberate.sql
Last active December 17, 2015 17:29
I wrote this as prep for a talk about JSON datatype and PLV8: https://speakerdeck.com/selenamarie/schema-liberation-with-json-and-plv8-and-postgres
CREATE SCHEMA liberated;
CREATE OR REPLACE FUNCTION public.liberate()
RETURNS boolean
LANGUAGE plv8
AS $function$
var tables = plv8.execute(
"select relname FROM pg_catalog.pg_class"
+ " c LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace"
@selenamarie
selenamarie / plv8x
Last active December 16, 2015 11:38
Installing plv8
# http://nonantolando.blogspot.tw/2012/10/using-json-on-ubuntu-1204-server-with.html
sudo apt-get install libv8-3.7.12.22 libv8-dev
# Install d8
# http://askubuntu.com/questions/141252/is-there-a-package-for-an-executable-version-of-v8
svn checkout http://v8.googlecode.com/svn/trunk/ ./v8
cd v8
make dependencies