likethesky/python history.txt

## python history.txt
Homebrew Community Discussions

https://discourse.brew.sh/t/multiple-config-scripts-and-potential-homebrew-software-installation-confusion/4921/11

fish2000
Alexander Böhn


2019 Jun 4

OK so: there are always personal, professional, stylistic, and inertial reasons to use one version of Python versus another. I’ll tell you briefly about how I personally see and use the macOS system Python, the Python.org Python, the various and sundry Homebrew Pythons, and a few others like Anaconda’s Python.

Fortunately, a given version of python – that is to say, a Python installation with a version number of x.y.z – will work the same as any given Python x.y.z install, provided they are both installed and invoked correctly. At least, as far as the core language and standard library goes… what packages each Python has available to it can and will vary greatly, unless you spend a lot of time ensuring that it doesn’t.

So with that caveat, let me explain. In the beginning, when I started programming in Python around 2005, I used the macOS system Python, which at the time was called the MacOS system Python (note the capitalization difference). The version was either 2.3 or 2.5, I can’t recall. The pip installer was not yet a thing – Python packaging was a terrific topic under which one could start a fantastic array of flamewars on listservs, or early Reddit or Digg (as StackOverflow was also not yet bethingified, and Hacker News was a small core cadre of sycophants). The executable was located at /usr/bin/python, and packages were installed to /Library/Python/2.5/site-packages. I didn’t know anything about Python stuff – I didn’t know why .egg files were terrible, or how native extensions were built and linked, or what the deal was with distutils and setuptools… or, most egregiously why you never ever want to mess around with Apple system binaries.

I used this Python because it shipped with PyObjC installed. PyObjC is a bridge between the Python and the Objective-C runtimes. It is very old (it has been in development since Steve Jobs was at NeXT), very thorough, and very vast – it’s a constellation of interdependant Python packages that, at the time, had to be compiled from source to be properly installed. Since I had a vested interest in using PyObjC and no idea how to install it from scratch, I took to using the system Python.

I will spare you all of the details of my subsequent learning-the-hard-way about meddling with Apple system binaries, and also forgo a lengthy digression into how I moved from trashing my MacOS installation to using package managers: first Fink, then MacPorts, and then finally Homebrew – which mercifully was the least insane and most pragmatically malleable of all of these. Between the epoch of MacPorts and the Homebrew era, I actually swore off MacOS package management for a while, and spent a brief lacuna working with the Python.org distribution, the well-known ActiveState releases, my own sub-optimal builds, and many others.

By this time, it was about 2011, and I had learned a few things: how using Homebrew’s Python for development would keep the macOS system Python clean and pristine; how virtualenv and virtualenvwrapper could sequester a project further, making it independent of the Python configuration; how testing and deployment with the Vagrant virtual-machine manager would keep everything in its right place – I have OCD, so these things all appealed to my innate organizational sense as well as my technical inclinations.

And the tools had grown as I had: pip had established itself as the de-facto Python package-management standard. A victor emerged from the packaging wars: setuptools was now being actively developed, instead of bickered over endlessly online. GitHub and git were more and more central every day to my development workflow, as well as that of countless others. Homebrew used git, and while it was written in Ruby, it was being developed by people who appreciated clean and legible code – if one could read the Homebrew source, one could hack on it.

This, then, is where I have come to stand on the various and sundry versions of Python that are available:

The macOS system Python – the Python install that ships with macOS

version: 2.7.10 (as of Mojave 10.14.5)
binary: /usr/bin/python
packages: /Library/Python/2.7/site-packages
verdict: DO NOT USE. This Python is an old and out-of-date version from the endangered 2.x lineage; even if you were somehow willing to deal with being locked into an old and un-upgraeable programming environment, you’d have to also take care not to disturb this Python installation, such that any macOS updates or hotfixes would be able to count on it working exactly as Apple assumes it would if it were pristine. That means you also can’t upgrade the bundled PyObjC packages, which at the time of writing are an appalingly ancient version, 2.5.1 (by contrast, I have PyObjC version 5.2b1 installed into my Homebrew Python’s package library). In fact, any upgrades, changes, tweaks, modifications, or ∆-minutiae you impose could screw things up later when some lazily-written Apple package script runs on the system Python and acts slightly different than it did in Cupertino. So yeah I repeat: DO NOT USE.
The official Python.org Python package distribution

version: latest stable, latest development RCs, and archives of prior releases
framework: /Library/Frameworks/Python.framework
binary: $framework + /Versions/3.7/bin
packages: $framework + /Versions/3.7/lib/python3.7/site-packages
verdict: Are you, or would you like to be, a core Python developer? Is contributing to the Python language and its central components – the parser, the interpreter, the bytecode generator, the peephole optimizer, the standard library, the “Argument Clinic” preprocessor, the Unicode algorithms, the POSIX wrappers – if you are still reading this and are specifically not bored, this is the Python package for you to use daily. Not to develop all of those non-boring things, no – you’ll of course want to have multiple git repositories of the Python source compiling and running in VMs orchestrated with Docker and whatnot for all that – but you, being a so-called “language lawyer,” will nonetheless want to work with the most official Python you can.
The Anaconda Python distribution – from the conda package manager

version: 3.7.3 and 2.7.16, respectively, at the time of writing
binary: /usr/local/conda/bin/python
packages: /usr/local/lib/python3.7/site-packages
verdict: Are you a data scientist? Or is that something you say you are when introducing yourself at meetups and conventions? Either way, if you love data and hate all those icky compiler flags and configuration files and settings directory paths and all that other nonsense, you might just be in the market for Anaconda. Anaconda, through its conda package-management CLU, is a whole ecosystem of data-science-y software – all of it easily and painlessly installable with just a few commands (like e.g. conda install numpy, conda install dlib)… Software originating from diverse ecosystems, with complex configurations and lengthy build-times, can all be installed easily in pre-tested, ready-to-run binary form through conda. Anaconda’s Python distribution is one of the fundamental components of this system, and it will always be built with the most reasonable options and delivered within a minimal lag from its official release. You, the data scientist par excellence, would clearly rather spend your time (and your processor’s time) training unsupervised models instead of compiling stuff – and that is why you, like Sir Mixalot, choose Anaconda!
The Homebrew Python distribution – from the python and python@2 formulae

version: 3.7.3 and 2.7.16, respectively, at the time of writing
framework:
Python 3: /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework
Python 2: /usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework
binary:
Python 3: $py3framework + /Versions/3.7/bin/python3.7
symlinked as /usr/local/bin/python3
Python 2: $py2framework + /Versions/2.7/bin/python2.7
symlinked as /usr/local/bin/python2
packages:
Python 3: /usr/local/lib/python3.7/site-packages
Python 2: /usr/local/lib/python2.7/site-packages
verdict: Use this Python for Python 3, and likely for Python 2. Homebrew has come a long way from its humble beginnings, as an @mxcl side-hustle that compiled everything you asked for and promiscuously linked with anything on your system – from SuperEnv isolation to bottle binary distribution to the stats-driven winnowing of superfluous options, Homebrew is pretty robust and not at all challenging to use. I use the Homebrew Python 3 installation as both my development Python and my workhorse Python – meaning it’s what I use to get miscellaneous tasks done around my system. While some might divide these two roles amongst two Pythons, I find that the one-off tools I write for workhorse tasks can benefit from some of my library development work, and vice versa. Homebrew Python installations have a great feature that installed packages like Python.org Pythons often lack: upgrading the Homebrew Python package will not clobber the associated site-packages directory, keeping your installed packages installed. In addition to Homebrew’s Python 3, I keep its Python 2 formula installed as well, mainly to satisfy the myriad requirements of the formulae that currently demand a Python 2 interpreter. I also occasionally fire up the Python 2 REPL to examine some quirk of the old environment, but I have to do that less and less frequently these days… I am champing at the bit for 2020, when Python 3 will be the Python!
The Homebrew PyPy distributions – from the pypy and pypy3 formulae

version: 7.1.1 and 7.0.0 – equivalent to 2.7.13 and 3.6.1, respectively
binary: /usr/local/bin/pypy and /usr/local/bin/pypy3
packages:
PyPy2: /usr/local/Cellar/pypy/7.1.1/libexec/site-packages
PyPy3: /usr/local/Cellar/pypy3/7.0.0/libexec/site-packages
verdict: Have you heard of PyPy? It’s kind of a crazy thing to describe it to those who haven’t: it’s a JIT-optimizing GIL-free reimplementation of the Python interpreter, written in a subset of Python 2 called “RPython”. PyPy is extremely fast, ridiculously internally convoluted, frustratingly counterintuitive to deal with, outrageously slow to compile (or “translate” as it is known in PyPy-land) and highly piquantly interesting. Luckily, you can install the bottled binaries from Homebrew and play around with it without tying up your computer for six or so hours (really). Internally, concepts implemented by PyPy include pickleable stack frames (from Stackless Python), a method JIT and a tracing JIT (from the JVM), a fair generational (non-refcounted) garbage-collection scheme (from everyone who ever tried to patch Python to remove the GIL), coroutines that are not hacked-up generator objects (from Go and Greenlet) and the async stuff from Python 3.4-plus-plus. Did you get all that? No? Don’t bother with PyPy, in that case – it will confuse the living daylights out of you. If you did get all that, and you have a codebase with no C-API extension dependencies, out of which you need to squeeze some performance gains before your next quarterly review so you can afford that time-share in Cleveland – then by all means, go for it, get yourself some PyPy and see what good can come of it. Otherwise DO NOT BOTHER unless you need to dispatch some serious free time and you don’t mind being flummoxed by a thing that deceptively looks like another thing with which you are familiar, but then isn’t. Yep.

Does that answer your question, @Kusine ? I do hope so, indeed!
	Homebrew Community Discussions

	https://discourse.brew.sh/t/multiple-config-scripts-and-potential-homebrew-software-installation-confusion/4921/11

	fish2000
	Alexander Böhn


	2019 Jun 4

	OK so: there are always personal, professional, stylistic, and inertial reasons to use one version of Python versus another. I’ll tell you briefly about how I personally see and use the macOS system Python, the Python.org Python, the various and sundry Homebrew Pythons, and a few others like Anaconda’s Python.

	Fortunately, a given version of python – that is to say, a Python installation with a version number of x.y.z – will work the same as any given Python x.y.z install, provided they are both installed and invoked correctly. At least, as far as the core language and standard library goes… what packages each Python has available to it can and will vary greatly, unless you spend a lot of time ensuring that it doesn’t.

	So with that caveat, let me explain. In the beginning, when I started programming in Python around 2005, I used the macOS system Python, which at the time was called the MacOS system Python (note the capitalization difference). The version was either 2.3 or 2.5, I can’t recall. The pip installer was not yet a thing – Python packaging was a terrific topic under which one could start a fantastic array of flamewars on listservs, or early Reddit or Digg (as StackOverflow was also not yet bethingified, and Hacker News was a small core cadre of sycophants). The executable was located at /usr/bin/python, and packages were installed to /Library/Python/2.5/site-packages. I didn’t know anything about Python stuff – I didn’t know why .egg files were terrible, or how native extensions were built and linked, or what the deal was with distutils and setuptools… or, most egregiously why you never ever want to mess around with Apple system binaries.

	I used this Python because it shipped with PyObjC installed. PyObjC is a bridge between the Python and the Objective-C runtimes. It is very old (it has been in development since Steve Jobs was at NeXT), very thorough, and very vast – it’s a constellation of interdependant Python packages that, at the time, had to be compiled from source to be properly installed. Since I had a vested interest in using PyObjC and no idea how to install it from scratch, I took to using the system Python.

	I will spare you all of the details of my subsequent learning-the-hard-way about meddling with Apple system binaries, and also forgo a lengthy digression into how I moved from trashing my MacOS installation to using package managers: first Fink, then MacPorts, and then finally Homebrew – which mercifully was the least insane and most pragmatically malleable of all of these. Between the epoch of MacPorts and the Homebrew era, I actually swore off MacOS package management for a while, and spent a brief lacuna working with the Python.org distribution, the well-known ActiveState releases, my own sub-optimal builds, and many others.

	By this time, it was about 2011, and I had learned a few things: how using Homebrew’s Python for development would keep the macOS system Python clean and pristine; how virtualenv and virtualenvwrapper could sequester a project further, making it independent of the Python configuration; how testing and deployment with the Vagrant virtual-machine manager would keep everything in its right place – I have OCD, so these things all appealed to my innate organizational sense as well as my technical inclinations.

	And the tools had grown as I had: pip had established itself as the de-facto Python package-management standard. A victor emerged from the packaging wars: setuptools was now being actively developed, instead of bickered over endlessly online. GitHub and git were more and more central every day to my development workflow, as well as that of countless others. Homebrew used git, and while it was written in Ruby, it was being developed by people who appreciated clean and legible code – if one could read the Homebrew source, one could hack on it.

	This, then, is where I have come to stand on the various and sundry versions of Python that are available:

	The macOS system Python – the Python install that ships with macOS

	version: 2.7.10 (as of Mojave 10.14.5)
	binary: /usr/bin/python
	packages: /Library/Python/2.7/site-packages
	verdict: DO NOT USE. This Python is an old and out-of-date version from the endangered 2.x lineage; even if you were somehow willing to deal with being locked into an old and un-upgraeable programming environment, you’d have to also take care not to disturb this Python installation, such that any macOS updates or hotfixes would be able to count on it working exactly as Apple assumes it would if it were pristine. That means you also can’t upgrade the bundled PyObjC packages, which at the time of writing are an appalingly ancient version, 2.5.1 (by contrast, I have PyObjC version 5.2b1 installed into my Homebrew Python’s package library). In fact, any upgrades, changes, tweaks, modifications, or ∆-minutiae you impose could screw things up later when some lazily-written Apple package script runs on the system Python and acts slightly different than it did in Cupertino. So yeah I repeat: DO NOT USE.
	The official Python.org Python package distribution

	version: latest stable, latest development RCs, and archives of prior releases
	framework: /Library/Frameworks/Python.framework
	binary: $framework + /Versions/3.7/bin
	packages: $framework + /Versions/3.7/lib/python3.7/site-packages
	verdict: Are you, or would you like to be, a core Python developer? Is contributing to the Python language and its central components – the parser, the interpreter, the bytecode generator, the peephole optimizer, the standard library, the “Argument Clinic” preprocessor, the Unicode algorithms, the POSIX wrappers – if you are still reading this and are specifically not bored, this is the Python package for you to use daily. Not to develop all of those non-boring things, no – you’ll of course want to have multiple git repositories of the Python source compiling and running in VMs orchestrated with Docker and whatnot for all that – but you, being a so-called “language lawyer,” will nonetheless want to work with the most official Python you can.
	The Anaconda Python distribution – from the conda package manager

	version: 3.7.3 and 2.7.16, respectively, at the time of writing
	binary: /usr/local/conda/bin/python
	packages: /usr/local/lib/python3.7/site-packages
	verdict: Are you a data scientist? Or is that something you say you are when introducing yourself at meetups and conventions? Either way, if you love data and hate all those icky compiler flags and configuration files and settings directory paths and all that other nonsense, you might just be in the market for Anaconda. Anaconda, through its conda package-management CLU, is a whole ecosystem of data-science-y software – all of it easily and painlessly installable with just a few commands (like e.g. conda install numpy, conda install dlib)… Software originating from diverse ecosystems, with complex configurations and lengthy build-times, can all be installed easily in pre-tested, ready-to-run binary form through conda. Anaconda’s Python distribution is one of the fundamental components of this system, and it will always be built with the most reasonable options and delivered within a minimal lag from its official release. You, the data scientist par excellence, would clearly rather spend your time (and your processor’s time) training unsupervised models instead of compiling stuff – and that is why you, like Sir Mixalot, choose Anaconda!
	The Homebrew Python distribution – from the python and python@2 formulae

	version: 3.7.3 and 2.7.16, respectively, at the time of writing
	framework:
	Python 3: /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework
	Python 2: /usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework
	binary:
	Python 3: $py3framework + /Versions/3.7/bin/python3.7
	symlinked as /usr/local/bin/python3
	Python 2: $py2framework + /Versions/2.7/bin/python2.7
	symlinked as /usr/local/bin/python2
	packages:
	Python 3: /usr/local/lib/python3.7/site-packages
	Python 2: /usr/local/lib/python2.7/site-packages
	verdict: Use this Python for Python 3, and likely for Python 2. Homebrew has come a long way from its humble beginnings, as an @mxcl side-hustle that compiled everything you asked for and promiscuously linked with anything on your system – from SuperEnv isolation to bottle binary distribution to the stats-driven winnowing of superfluous options, Homebrew is pretty robust and not at all challenging to use. I use the Homebrew Python 3 installation as both my development Python and my workhorse Python – meaning it’s what I use to get miscellaneous tasks done around my system. While some might divide these two roles amongst two Pythons, I find that the one-off tools I write for workhorse tasks can benefit from some of my library development work, and vice versa. Homebrew Python installations have a great feature that installed packages like Python.org Pythons often lack: upgrading the Homebrew Python package will not clobber the associated site-packages directory, keeping your installed packages installed. In addition to Homebrew’s Python 3, I keep its Python 2 formula installed as well, mainly to satisfy the myriad requirements of the formulae that currently demand a Python 2 interpreter. I also occasionally fire up the Python 2 REPL to examine some quirk of the old environment, but I have to do that less and less frequently these days… I am champing at the bit for 2020, when Python 3 will be the Python!
	The Homebrew PyPy distributions – from the pypy and pypy3 formulae

	version: 7.1.1 and 7.0.0 – equivalent to 2.7.13 and 3.6.1, respectively
	binary: /usr/local/bin/pypy and /usr/local/bin/pypy3
	packages:
	PyPy2: /usr/local/Cellar/pypy/7.1.1/libexec/site-packages
	PyPy3: /usr/local/Cellar/pypy3/7.0.0/libexec/site-packages
	verdict: Have you heard of PyPy? It’s kind of a crazy thing to describe it to those who haven’t: it’s a JIT-optimizing GIL-free reimplementation of the Python interpreter, written in a subset of Python 2 called “RPython”. PyPy is extremely fast, ridiculously internally convoluted, frustratingly counterintuitive to deal with, outrageously slow to compile (or “translate” as it is known in PyPy-land) and highly piquantly interesting. Luckily, you can install the bottled binaries from Homebrew and play around with it without tying up your computer for six or so hours (really). Internally, concepts implemented by PyPy include pickleable stack frames (from Stackless Python), a method JIT and a tracing JIT (from the JVM), a fair generational (non-refcounted) garbage-collection scheme (from everyone who ever tried to patch Python to remove the GIL), coroutines that are not hacked-up generator objects (from Go and Greenlet) and the async stuff from Python 3.4-plus-plus. Did you get all that? No? Don’t bother with PyPy, in that case – it will confuse the living daylights out of you. If you did get all that, and you have a codebase with no C-API extension dependencies, out of which you need to squeeze some performance gains before your next quarterly review so you can afford that time-share in Cleveland – then by all means, go for it, get yourself some PyPy and see what good can come of it. Otherwise DO NOT BOTHER unless you need to dispatch some serious free time and you don’t mind being flummoxed by a thing that deceptively looks like another thing with which you are familiar, but then isn’t. Yep.

	Does that answer your question, @Kusine ? I do hope so, indeed!