Skip to content

Instantly share code, notes, and snippets.

Avatar

Nick Zadrozny nz

View GitHub Profile
View help me with nix.md

I really want to love Nix!

The concepts and the architecture are compelling. It resonates strongly with so much of my own values, based on now 20 years of programming experience, and a solid decade of large-scale operational engineering. (I manage large fleets of Solr and Elasticsearch search engines.)

The small amount of play with Nix, and the medium amount of reading I've done are encouraging. I can get some packages installed. I can start a toy nix-shell with some language or other present. I can read a Nix derivation and pretty much follow along with what's happening, although I am far from fluent with writing the Nix language.

But right now I'm hitting a wall when it comes to a more complex real-world use case.

  1. Create a pure and isolated development environment for a Rails app, using Postgres.
  2. Create a pure and isolated development simple Crystal app.
@nz
nz / csv-usage-example.rb
Last active Sep 3, 2019
Dynamic time-based batch sizing
View csv-usage-example.rb
elasticsearch_url = ENV.fetch('ELASTICSEARCH_URL', 'http://localhost:9200')
elasticsearch = Elasticsearch::Client.new(url: elasticsearch_url, trace: true)
importer = Importer.new
importer.batch_handler = lambda do |actions|
elasticsearch.bulk(body: actions)
end
importer.start
csv = CSV.new(File.open('data/books.csv', 'r'), headers: true)
@nz
nz / cfn.toml
Last active Jun 1, 2016
Because why not
View cfn.toml
AWSTemplateFormatVersion = "2010-09-09"
Description = "AWS CloudFormation Sample Template vpc_multiple_subnets.template: Sample template showing how to create a VPC with multiple subnets. The first subnet is public and contains the load balancer, the second subnet is private and contains an EC2 instance behind the load balancer. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template."
[Mappings.AWSInstanceType2Arch.c1.medium]
Arch = "64"
[Mappings.AWSInstanceType2Arch.c1.xlarge]
Arch = "64"
[Mappings.AWSInstanceType2Arch.m1.large]
Arch = "64"
[Mappings.AWSInstanceType2Arch.m1.medium]
Arch = "64"
@nz
nz / oauth lite.md
Last active Jun 8, 2016
Light weight HMAC token auth over HTTP Basic Auth
View oauth lite.md

HMAC over Basic Auth

This is a pattern I use fairly frequently for administrative APIs. It's a sort of OAuth lite for non-public APIs that produces good quality tokens. Once you build it a few times, it's not any harder than using arbitrary basic auth in your APIs.

The client and the app share a secret, which is never transmitted across the wire. The client uses this secret to create an HMAC digest of a payload consisting of the current time and a random nonce value. The nonce is provided as the Basic Authorization user, and the resulting HMAC digest is provided as the Basic Authorization password.

A similar process is followed on the server side. The server uses the supplied nonce, its own time, and its own copy of the shared secret. It may want to check against several tokens across a small window of times to account for clock drift.

  • Using HMAC means the secret is never transmitted across the wire. Theoretically these are safe across plaintext connections, but you're using TLS anyway, right?
  • The i
@nz
nz / rsolr_with_default_headers.rb
Last active Jan 21, 2016
RSolr with default headers for websolr preferential routing to master or slave
View rsolr_with_default_headers.rb
# Websolr can use HTTP headers to control authorization and request routing.
# This class injects a connection wrapper into Sunspot to set default headers
# on every request.
#
# Interesting values for X-Websolr-Routing:
# - prefer-master (route serches to master, for real-time search; cpu expensive; current default)
# - prefer-random (distribute searches evenly; for high-volume of searches relative to updates)
# - prefer-replica (isolate search requests from updates; for high-volumes of one type of traffic negatively impacting the other)
#
@nz
nz / elasticsearch-term-frequency.sh
Created Dec 15, 2014
In Elasticsearch, how do I get a list of the top 10 most frequently occurring values of a field?
View elasticsearch-term-frequency.sh
#!/bin/sh
test_document="{
\"text\": \"HEAR the sledges with the bells, / Silver bells! / What a world of merriment their melody foretells! / How they tinkle, tinkle, tinkle, / In the icy air of night! / While the stars, that oversprinkle / All the heavens, seem to twinkle / With a crystalline delight; / Keeping time, time, time, / In a sort of Runic rhyme, / To the tintinnabulation that so musically wells / From the bells, bells, bells, bells, / Bells, bells, bells— / From the jingling and the tinkling of the bells.\"
}"
if curl -fs -X HEAD localhost:9200/top-terms; then
echo "Clear the old test index"
curl -X DELETE localhost:9200/top-terms; echo "\n"
fi
View elasticsearch-haystack-bonsai.py
ES_URL = urlparse(os.environ.get('BONSAI_URL') or 'http://127.0.0.1:9200/')
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': ES_URL.scheme + '://' + ES_URL.hostname + ':80',
'INDEX_NAME': 'haystack',
},
}
@nz
nz / irbrc-rails-env.rb
Created May 29, 2014
My ~/.irbrc with the Rails app name and environment. A useful bit of paranoia to help prevent operator errors.
View irbrc-rails-env.rb
if defined?(Rails) && Rails.env
reset = "\e[0m"
color = case Rails.env
when 'development', 'test'
"\e[36m" # cyan
else
"\e[31m" # magenta
end
View keybase.md

Keybase proof

I hereby claim:

  • I am nz on github.
  • I am nz (https://keybase.io/nz) on keybase.
  • I have a public key whose fingerprint is 9B6A A415 178D EE9F A265 B3E9 DF54 6078 26E3 B167

To claim this, I am signing this object:

@nz
nz / gist:8852934
Created Feb 6, 2014
Rails Concern to create a bunch of downcased getter methods
View gist:8852934
module Downcaseable
extend ActiveSupport::Concern
module ClassMethods
def downcase_field(*names)
Array(names).flatten.each do
class_eval %Q(
def #{name}_eval
send(:#{name}).try(:downcase)
end
)