Skip to content

Instantly share code, notes, and snippets.

@shevron
shevron / make-test-csv.py
Last active November 20, 2020 07:10
Generate an arbitrary CSV file of about given size in Python
"""Use to generate a large CSV file for testing purposes. Will write CSV data to STDOUT
Usage: python make-test-csv.py 512000000 # Make a ~512mb CSV file
"""
import csv
import sys
from itertools import count
def make_csv_row(counter, row_width, zero_fill=8):
@shevron
shevron / azure-uploader.py
Created October 6, 2020 12:01
Demo of using Azure Blob Storage API for simple multipart upload flow
"""Test Azure Multipart Uploads
"""
import base64
import logging
import os
from itertools import count
from typing import BinaryIO, Generator, Tuple
import click
from azure.core.exceptions import ResourceNotFoundError
@shevron
shevron / docker-compose-dev.yaml
Created April 9, 2019 06:18
Docker compose file defining external CKAN dependencies
# Docker compose for external services, assuming main ckan services are run
# outside of Docker
version: "3"
volumes:
pg_data:
services:
@shevron
shevron / oauth2_key_example.sh
Created October 23, 2016 06:20
Shoppimon API Examples (bash / curl / jq)
# The following example requires `jq` to parse JSON
CLIENT_ID=abcdef12345678s
CLIENT_SECRET=0123456789abcdef0123456789abcdef
BEARER_TOKEN=$(curl -s -X POST https://api.shoppimon.com/oauth \
-H "Content-Type: application/json" \
-d '{"grant_type":"client_credentials","client_id":"'$CLIENT_ID'","client_secret":"'$CLIENT_SECRET'"}' |\
jq -rM '.access_token')
export BEARER_TOKEN
echo "Bearer token: $BEARER_TOKEN"
@shevron
shevron / code_with_locking_context.py
Last active April 3, 2019 15:08
Examples: MySQL named locks and Python context managers
"""same logic as in above, but using a locking context manager
"""
from app_services import db_session
from locking_context import named_lock
def single_running_func():
"""this code should never execute in parallel
"""
with named_lock(db_session, 'my_lock', 5):
# Do some stuff that should not run in parallel
@shevron
shevron / test.php
Last active August 29, 2015 14:01 — forked from anonymous/test.php
A better way to write to /dev/null
<?php
// Get 10mb of data from /dev/zero
$infp = fopen('/dev/zero', 'r');
$data = fread($infp, 1024 * 1024 * 10);
fclose($infp);
write_data($data);
echo "Peak memory usage: ", number_format(memory_get_peak_usage()), " bytes\n";
@shevron
shevron / LICENSE
Last active April 28, 2020 02:28
Send EC2 instance memory usage stats to CloudWatch using boto and IAM Roles
Copyright (c) 2015, Shahar Evron
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
@shevron
shevron / builtin-ws-wrapper.php
Created August 18, 2012 09:28
A wrapper script for serving ZF apps with PHP's built in Web server
<?php
/**
* PHP Built-in Web Server wrapper for ZF apps
*/
$reqUri = $_SERVER['REQUEST_URI'];
if (strpos($reqUri, '?') !== false) {
$reqUri = substr($reqUri, 0, strpos($reqUri, '?'));
}
@shevron
shevron / fix-cs-multiuse.php
Created August 12, 2012 10:45
Convert single, comma separated mutli-namspace use statements into multiple use calls with single namespace references
<?php
/**
* Convert single use calls with comma separated list of namespaces into
* multiple, stand-alone use calls
*
* Usage: php cs-fix-use.php <file>
*
* Will overwrite <file> with new version. If <file> is omitted, stdin / stdout
* will be used.
@shevron
shevron / Uri.php
Created June 18, 2012 18:18
Zend Framework 2.0 Uri validator
<?php
namespace Zend\Validator;
class Uri extends AbstractValidator
{
const INVALID = 'uriInvalid';
protected $_messageTemplates = array(
self::INVALID => "Provided input is not a valid URL"