Skip to content

Instantly share code, notes, and snippets.

💭
hustlin'

Stephan Fitzpatrick knowsuchagency

💭
hustlin'
  • San Francisco
Block or report user

Report or block knowsuchagency

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@simonw
simonw / recover_source_code.md
Last active Nov 13, 2019
How to recover lost Python source code if it's still resident in-memory
View recover_source_code.md

How to recover lost Python source code if it's still resident in-memory

I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6

Attach a shell to the docker container

Install GDB (needed by pyrasite)

apt-get update && apt-get install gdb
@inchoate
inchoate / including_external_package_in_dataflow.md
Last active Aug 20, 2019
Adding an extra package to a Python Dataflow project to run on GCP
View including_external_package_in_dataflow.md

The Problem

The documentation for how to deploy a pipeline with extra, non-PyPi, pure Python packages on GCP is missing some detail. This gist shows how to package and deploy an external pure-Python, non-PyPi dependency to a managed dataflow pipeline on GCP.

TL;DR: You external package needs to be a python (source/binary) distro properly packaged and shipped alongside your pipeline. It is not enough to only specify a tar file with a setup.py.

Preparing the External Package

Your external package must have a proper setup.py. What follow is an example setup.py for our ETL package. This is used to package version 1.1.1 of the etl library. The library requires 3 native PyPi packages to run. These are specified in the install_requires field. This package also ships with custom external JSON data, declared in the package_data section. Last, the setuptools.find_packages function searches for all available packages and returns that

@peterhurford
peterhurford / pytest-fixture-modularization.md
Created Jul 28, 2016
How to modularize your py.test fixtures
View pytest-fixture-modularization.md

Using py.test is great and the support for test fixtures is pretty awesome. However, in order to share your fixtures across your entire module, py.test suggests you define all your fixtures within one single conftest.py file. This is impractical if you have a large quantity of fixtures -- for better organization and readibility, you would much rather define your fixtures across multiple, well-named files. But how do you do that? ...No one on the internet seemed to know.

Turns out, however, you can define fixtures in individual files like this:

tests/fixtures/add.py

import pytest

@pytest.fixture
@stinger
stinger / Swift3GCDURLSessionDataTask.swift
Last active Mar 26, 2018
Swift 3: GCD and URLSessionDataTask with a completionHandler
View Swift3GCDURLSessionDataTask.swift
//: # Swift 3: CGD and URLSessionDataTask
import Foundation
import PlaygroundSupport
//: ### Create background queue
let queue = DispatchQueue.global(qos: .background)
//: ### Computed variable
var time:DispatchTime! {
@jbn
jbn / aiohttp_uptime_http_server.py
Created Apr 7, 2016
An example showing how to stream HTML in a aiohttp server.
View aiohttp_uptime_http_server.py
import asyncio
from aiohttp import web
import subprocess
async def uptime_handler(request):
# http://HOST:PORT/?interval=90
interval = int(request.GET.get('interval', 1))
# Without the Content-Type, most (all?) browsers will not render
@lukas-h
lukas-h / license-badges.md
Last active Nov 14, 2019
Markdown License Badges for your Project
View license-badges.md

Markdown License badges

Collection of License badges for your Project's README file.
This list includes the most common open source and open data licenses.
Easily copy and paste the code under the badges into your Markdown files.

Notes

@olivierlacan
olivierlacan / migrate_postgresql_database.md
Last active Nov 14, 2019
How to migrate a Homebrew-installed PostgreSQL database to a new major version (9.3 to 9.4) on OS X. See upgraded version of this guide: http://olivierlacan.com/posts/migrating-homebrew-postgres-to-a-new-version/
View migrate_postgresql_database.md

This guide assumes that you recently run brew upgrade postgresql and discovered to your dismay that you accidentally bumped from one major version to another: say 9.3.x to 9.4.x. Yes, that is a major version bump in PG land.

First let's check something.

brew info postgresql

The top of what gets printed as a result is the most important:

@gdamjan
gdamjan / client.py
Last active Nov 10, 2019
Python 3.5 async/await with aiohttp, parallel or sequential
View client.py
import aiohttp
import asyncio
async def get_body(url):
response = await aiohttp.request('GET', url)
raw_html = await response.read()
return raw_html
async def main():
# run them sequentially (but the loop can do other stuff in the meanwhile)
@swlaschin
swlaschin / ConstrainedTypesExamples.fsx
Last active Nov 9, 2019
Examples of creating constrained types in F#
View ConstrainedTypesExamples.fsx
// General hints on defining types with constraints or invariants
//
// Just as in C#, use a private constructor
// and expose "factory" methods that enforce the constraints
//
// In F#, only classes can have private constructors with public members.
//
// If you want to use the record and DU types, the whole type becomes
// private, which means that you also need to provide:
// * a constructor function ("create").
You can’t perform that action at this time.