wickman

Python development using Pants

brian wickman - @wickman

[TOC]

Why use Pants for Python development?

Pants makes the manipulation and distribution of hermetically sealed Python environments

wickman

from abc import ABCMeta, abstractmethod
import threading


class ClockInterface(object):
  __metaclass__ = ABCMeta

  @abstractmethod
  def time(self):
    pass

wickman

from string import Formatter


class RecursiveFormatter(Formatter):
  def _contains_underformatted_field_names(self, format_tuple):
    literal_text, field_name, format_spec, conversion = format_tuple
    if field_name is not None:
      return any(component[1] is not None for component in self.parse(field_name))
    return False

wickman

Installing / troubleshooting pants

Requirements

Most of the commons python environment has been developed against CPython 2.6. Things mostly work with CPython 2.7 and recent efforts have been made to improve CPython 3.x and PyPy compatibility. We've explicitly ignored anything prior to CPython 2.6 and in fact generally discourage use against anything less than CPython 2.6.5 as there are known bugs that we're unwilling to fix. We've never even tried running against Jython or

wickman

# E.g. TaskClass = CaseClass('name', 'owner', 'pid')
# task1 = TaskClass(name = "hello", owner = "brian", pid = 15)
# task2 = TaskClass(name = "world", owner = "brian", pid = 13)
# tasks = [task1, task2]
# 
# filter(lambda task: task.where(owner = "brian"), tasks) => [task1, task2]
# filter(lambda task: task.where(owner = "brian", pid = 13), tasks) => [task2]
# 
# matcher = TaskClass(pid = 13)
# filter(lambda task: task.match(matcher), tasks) => [task2]

wickman

Pystachio 1.0 redesign



1. Documents:

Essentially a set of hierarchical key/value pairs a la a JSON document.

Documents may contain:
  1. Leaves (string or numeric [integer, long, float, etc])

wickman

Pex.pex: Usage

PEX files are single-file lightweight virtual Python environments.

pex.pex is a utility that:

• creates PEX files
• provides a single use run-environment

Installation

wickman

mba=science=; PANTS_VERBOSE=1 ./pants src/python/twitter/aurora/client/bin:aurora_client
Build operating on targets: OrderedSet([PythonBinary(src/python/twitter/aurora/client/bin/BUILD:aurora_client)])
Building PythonBinary PythonBinary(src/python/twitter/aurora/client/bin/BUILD:aurora_client):
Building PythonBinary PythonBinary(src/python/twitter/aurora/client/bin/BUILD:aurora_client):
  Dumping library: PythonLibrary(src/python/twitter/aurora/client/bin/BUILD:aurora_client_source)
  Dumping library: PythonLibrary(src/python/twitter/common/app/BUILD:app)
  Dumping library: PythonLibrary(src/python/twitter/common/BUILD:common)
  Dumping library: PythonLibrary(src/python/twitter/common/collections/BUILD:collections)
  Dumping library: PythonLibrary(src/python/twitter/common/config/BUILD:config)
  Dumping library: PythonLibrary(src/python/twitter/common/lang/BUILD:lang)

wickman

INSTALL_ROOT=$HOME/Python
CPY=$INSTALL_ROOT/CPython
PYPY=$INSTALL_ROOT/PyPy

SANDBOX=$(mktemp -d /tmp/python.XXXXXX)

CURL='wget --no-check-certificate'

mkdir -p $INSTALL_ROOT

wickman

from abc import ABCMeta, abstractmethod

def documented_by(documenting_abc):
  def document(cls):
    cls_dict = cls.__dict__.copy()
    for attr in documenting_abc.__abstractmethods__:
      cls_dict[attr].__doc__ = getattr(documenting_abc, attr).__doc__
    return type(cls.__name__, cls.__mro__[1:], cls_dict)
  return document