Skip to content

Instantly share code, notes, and snippets.

@datagrok datagrok/
Last active Mar 12, 2019

What would you like to do?
What's the best way to write function pipelines in Python?

I agree with Guido, in that I think

[ x * 2 for x in some_list if x < 12 ]

is much easier to read than the equivalent

map(lambda x:x * 2, filter(lambda x:x<12, some_list))

But what if you have several functions? In Haskell,

[ (f3 . f2 . f1) x | x <- some_list, x < 12]

seems easier to read than

map (f3 . f2 . f1) $ filter (< 12) some_list

but composition feels backwards compared to the pipeline-style I'd use when writing in in Python:

result = (f1(x) for x in some_list if x < 12)
result = (f2(x) for x in result)
result = (f3(x) for x in result)

But with neither compose nor pipeline operators, this is repetitious and verbose, among other flaws.

@yedpodtrzitko mentions 0101/pipetools which looks much better to me:

from pipetools import pipe
result = [ x > pipe | f1 | f2 | f3 for x in some_list if x < 12]

Unfortunately, this requires a non-standard library. But other languages include such a feature as a built-in. Elixir:

result = for x <- some_list, x < 12, do: x |> f1 |> f2 |> f3


import Data.Function
[ x & f1 & f2 & f3 | x <- some_list, x < 12]

So maybe I shouldn't feel awkward about using this library everywhere I like, with the rationale that such a useful tool really should be in the standard library anyway.


This comment has been minimized.

Copy link

yedpodtrzitko commented Sep 8, 2017

Hi, I stumbled upon this by accident, but maybe you want to check pipetools:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.