how to build a binary version of a Python package and make it pure binary

README.md

How to build a binary version of a pure Python package and make it pure binary

The package

Let we have a package that looks this way:

• mext/ package directory
  • __init__.py package "constructor" module
  • m1.py package member module #1
  • m2.py package member module #2
  • m3.py package member module #3
  • m4.py package member module #4

We want to have it in binary (*.so) for some obscure reasons.

setup.py

First off we have to write that setup.py.

It doesn't matter much what else it may do, but it must have these imports:

from distutils.core import setup # because it does everithing
from distutils.command import bdist_rpm # because we want an RPM output
from distutils.extension import Extension # because we build it as an "extension"
from Cython.Distutils import build_ext # because it must be "real binary" thing

In the args of setup() call we specify

    'cmdclass':  {'build_ext': build_ext}, # to make cython to join the game

The build

Run the create-rpm.sh to get dist/mext-0.0-1.x86_64.rpm file.

If you take a look (rpm -qlp $RPM) at the fresh build, then you'll see all those *.py, *.pyc, and *.pyo files you want to get rid of.

so,

The fix

Run the repack-rpm.sh.

It will

1. create sorta .spec file from the original RPM
2. unpack the content of that RPM
3. remove all the *.py* there
4. add an empty __init__.py file for each and every __init__.so one (to keep it package)
5. build it back using that .spec file from the step 1

Now you may copy the RPM to the target system and install using sudo rpm -ivh --force $RPM.

Why --force? Because its intrinsic cpio wanna create directories like /usr/local...
It looks harmless, anyway.

Here we are, folks.

create_rpm.sh

#!/bin/bash

[ -f "setup.py" ] || { echo "No setup.py here.">&2; exit 1; }

[ -d build ] && rm -rf build
[ -d dist ] && rm -rf dist

python setup.py bdist_rpm || { echo "Cannot build RPM.">&2; exit 1; }

ls -la dist

# vim: set ts=4 sts=4 sw=4 ai et : #

repack-rpm.sh

#!/bin/bash

RPM='dist/mext-0.0-1.x86_64.rpm'
name=$(basename "$RPM" .rpm)
pdir='/usr'

make_spec() {
    rpm -qpi "$@" \
    | grep -Ev '^(Architecture|Install Date|Group|Size|Signature|Source RPM)[ \t]*:' \
    | grep -Ev '^(Build Date|Build Host|Relocations)[ \t]*:' \
    | sed -e 's/^Description.*$/BuildRoot: .\n%description/'
    echo '%files'
    echo "$pdir/*"
}

[ -f "$RPM" ] || { echo "No RPM in '$RPM'.">&2; exit 1; }

# make build root tree
bdir=$(mkdir -pv ~/rpmbuild/BUILDROOT/$name >&2 && cd ~/rpmbuild/BUILDROOT/$name >&2 && pwd)
[ -z "$bdir" ] && { echo "Cannot make build root.">&2; exit 1; }

# create a [fake] spec file
make_spec "$RPM" > "$RPM.spec" || { echo "Cannot make spec for '$RPM'.">&2; exit $?; }

# unpack the RPM
{ rpm2cpio "$RPM" | ( cd "$bdir" && cpio -i --make-directories && ls -la; ); } \
    || { echo "Cannot unpack '$RPM'.">&2; exit $?; }

# hack your RPM here
echo "Updating..."
## remove source/compiled files
( cd "$bdir" && find . -type f -name '*.py*' -exec rm -v {} \; ; ) \
    || { echo "Cannot remove python sources from '$RPM'.">&2; exit $?; }
for so in $(find "$bdir" -type f -name __init__.so -print); do
    echo "+ '${so%.so}.py'" >&2
    touch "${so%.so}.py" || { echo "Cannot touch .py for '$so'."; exit $?; }
done

# pack it back
rpmbuild -v -bb "$RPM.spec" || { echo "Cannot build RPM for '$RPM'.">&2; exit $?; }

[ -f ~/rpmbuild/RPMS/x86_64/$name.rpm ] || { echo "Cannot find results.">&2; exit 1; }

ls -la ~/rpmbuild/RPMS/x86_64/$name.rpm || exit $?
rpm -qpl ~/rpmbuild/RPMS/x86_64/$name.rpm || exit $?

mv -v "$RPM" "$RPM".was || exit $?
mv -v ~/rpmbuild/RPMS/x86_64/$name.rpm "$RPM"

# vim: set ts=4 sts=4 sw=4 ai et : #

setup.py

import sys
import os
from distutils.core import setup
from distutils.command import bdist_rpm
from distutils.extension import Extension
from Cython.Distutils import build_ext
from Cython.Build import cythonize

extensions = [
    Extension(name='mext.__init__', sources=['mext/__init__.py']),
    Extension(name='mext.m1', sources=['mext/m1.py']),
    Extension(name='mext.m2', sources=['mext/m2.py']),
    Extension(name='mext.m3', sources=['mext/m3.py']),
    Extension(name='mext.m4', sources=['mext/m4.py']),
]

args = dict(
    name = 'mext',
    packages = ['mext'],
    ext_modules = extensions,
    cmdclass = {'build_ext': build_ext},

    version = '0.0',
    description = 'Attempt to build a binary distro',
    long_description = 'I wanna have a relatively robust file that end user may find hard to corrupt',
    author = 'jno',
    author_email = 'jno@work',
    maintainer = 'unmaintained',
    url = 'file:///dev/null',
    download_url = 'file:///dev/zero',
    classifiers = [
        'Development Status :: 3 - Alpha',
        'Programming Language :: Python :: 2.7',
        'Topic :: Software Development :: Libraries'
    ],

     #install_requires = ['cython'],
     options = dict(
        install = dict(
            prefix = '/opt/intra', # it will add 'lib/python2.7/site-packages' automagically
        ),
     ),
)

if len(sys.argv) < 2:
    args['script_args'] = ['build_ext']

setup(**args)