Skip to content

Instantly share code, notes, and snippets.

Avatar

Dan Blanchard dan-blanchard

View GitHub Profile
@letmaik
letmaik / README.md
Last active May 10, 2019
Using travis_retry inside shell scripts together with set -e
View README.md

If you want to use travis_retry from within your own shell script files then you first have to make the travis_retry shell function available by sourcing the travis_retry.sh file, otherwise you just get a "command not found" error. See example.sh for a full example.

Note that the original function as found in the travis-ci/travis-build repository was slightly modified to allow it to be used in a shell context where set -e is enabled.

For reference, a tweet by Travis CI saying that you should copy the travis_retry code as I've done here: https://twitter.com/plexus/status/499194992632811520

View postmates.py
#!/usr/bin/env python
from bs4 import BeautifulSoup
from twilio.rest import TwilioRestClient
import json
import os
import re
import requests
url = 'https://postmates.com/los-angeles'
@saranrapjs
saranrapjs / default.vcl
Last active Feb 15, 2018
Varnish + JSONP example
View default.vcl
vcl 4.0;
backend default {
.host = "127.0.0.1";
.port = "80";
}
sub vcl_recv {
unset req.http.callback;
@xofer
xofer / 0-storm-proxy-readme.md
Last active Mar 21, 2018
Storm UI Nginx conf for Nimbus with logviewer proxy
View 0-storm-proxy-readme.md

We have a setup that I assume is quite common: A publicly accessible Nimbus running Storm UI. The worker nodes can only be accessed from the Nimbus (via the LAN). All the nodes have internal DNS names (i.e. node.lan.example.com), which is set in the configuration files; they use these DNS names to reach each other. The Nimbus has an external DNS name (storm.example.com) for public access. The Nimbus's UI is behind an Nginx proxy, which provides HTTP Auth and HTTPS.

Because of this setup, the logviewer links in the UI do not work. In order to fix this, we employ an elaborate hack shown in the conf file below. It uses ngx_http_substitutions_filter_module to rewrite content returned by the Storm UI and some complicated URL rewrite tricks to proxy the workers' logviewers through through the Nimbus.

@topiaruss
topiaruss / pytest.ini
Last active Jun 21, 2016
Experimenting with minimal test frameworks for spout and bolt
View pytest.ini
[pytest]
python_paths = src/bolts src/spouts
testpaths = src/tests
View pygil.stp
# stap -g pygil.stp <pid> <minlocktime-in-mus>
global init = 0
global gpid = $1
global lock_start = 0
global mintime = $2
# may need to be libpython on some systems
probe process("/usr/bin/python-dbg").function("PyThread_acquire_lock")
{
if (init == 0) {
@sguillope
sguillope / bamboo_delete_disabled_branches.sh
Last active Jan 21, 2020
Script to delete all disabled branches of a Bamboo build plan.
View bamboo_delete_disabled_branches.sh
#!/bin/bash -u
# Deletes all disabled branches of a Bamboo build plan
# -----------------------------------------------------------------------------
# Syntax:
# $0 {planKey}
# -----------------------------------------------------------------------------
# Purpose: Bamboo does not automatically delete plan branches when the
# corresponding branch in the repository gets deleted. Because Bamboo fails
# to pull from it, it disables the branch but keep it around forever.
# This script goes through all branches of a build plan and delete the ones
View argparse_reminder.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser(description="""
I never freaking remember argparse syntax and the docs are so all over the place
that I need this for an example.
@dan-blanchard
dan-blanchard / .1.miniconda.md
Last active Dec 11, 2019
Quicker Travis builds that rely on numpy and scipy using Miniconda
View .1.miniconda.md

For ETS's SKLL project, we found out the hard way that Travis-CI's support for numpy and scipy is pretty abysmal. There are pre-installed versions of numpy for some versions of Python, but those are seriously out of date, and scipy is not there are at all. The two most popular approaches for working around this are to (1) build everything from scratch, or (2) use apt-get to install more recent (but still out of date) versions of numpy and scipy. Both of these approaches lead to longer build times, and with the second approach, you still don't have the most recent versions of anything. To circumvent these issues, we've switched to using Miniconda (Anaconda's lightweight cousin) to install everything.

A template for installing a simple Python package that relies on numpy and scipy using Miniconda is provided below. Since it's a common s

You can’t perform that action at this time.