topic-style-pipeline.md

Topic-Style Pipelines

Note: This is a distillation of @jschoi's excellent and thorough "Smart Mix Pipeline" proposal. @jschoi's proposal is perhaps too thorough, tho, and includes a number of optional features that I think muddy the waters and make the whole thing seem far more complex than it is.

So here's the minimal "topic-style pipeline" syntax I think we need, along with the two most important optional extensions only.

• Topic-style pipeline
  • Optional (mildly desired) feature: bare-style syntax
  • Optional (strongly desired) feature: pipeline function
    • Required subfeature: additional topic identifiers
  • Optional (mildly desired) restriction: class and function "hide" the topic reference

---

The reason I stan so hard for topic-style pipeline is its inherent simplicity. This simplicity manifests in two very important (and interlinked) ways, that no other pipeline proposal has:

1. The pipeline body is just an expression to be evaluated. There are no rules or restrictions on what it can be, no reinterpretation of it; the only thing distinguishing it from a non-pipeline expression is that it has a lexical scope wrapped around it that binds the topic reference (the # character) to the result of the pipeline head.

   This is very important; it means that the pipeline body automatically works with all of JS, past, present, and future. Anything we invent, any new syntax or new concepts we introduce, automatically work in a topic-style pipeline body.

   (In contrast, functional-style requires the pipeline body to be a function, or wrapped in a function. This means that syntax which is aware of function wrappers being introduced, such as await or yield, simply doesn't work, and needs to be specially addressed with unique syntax forms, making the pipeline syntax more complex.)

2. When unwrapping a complex expression into a pipeline, the entire process is:

   1. Cut out the complicated inner bit that you think of as a separate "step" in the expression.
   2. Paste it before the expression, and add a |> between them.
   3. Put in a # in the original expression where you removed the bit.

   That's it! That's the process 100% of the time, no matter what the original expression was, or what you removed. It's perfectly consistent every single time, and gives you complete flexibility in where you want to slice the original expression apart.

   (In contrast, functional-style shares the first two steps, but then for the third, you instead first check if the leftover expression was an unary function that you removed the argument of; if so, you delete the leftover (); otherwise, you wrap it in an arrow function and replace the removed bit with the arrow function's argument. Unless the leftover expression has an await or yield, then you need to do further transforms to put the await/yield in its own pipeline body so it can be handled as a special form; this is required even if you conceptually prefer to think of the await as a part of a larger unitary "step".)

   ---

   (2), above, is phrased in terms of transforming a non-piped expression into a piped one. But it applies more widely, which is concerning; it infects editing existing code.

   With topic-style syntax, authors who want to edit one of the pipeline bodies just... do so:

   val |> foo(#) |> bar(#)
   // hmm, actually I need to +1 the result of the foo step...
   val |> foo(#)+1 |> bar(#)
   

   With functional-style, you need to recognize which syntax your body is in, and possibly translate it to a different syntax in order to make the edit.

   val |> foo |> bar
   // hmm, actually I need to +1 the result of the foo step...
   val |> foo+1 |> bar // error!
   // need to transform the "foo" step into a lambda first...
   val |> (x=>foo(x)) |> bar
   // now I can +1 it
   val |> (x=>foo(x)+1) |> bar
   

   This isn't great as it is, but it gets worse: you still need to further recognize when you're about to transform it into using await, and do a different transform:

   val |> foo |> bar
   // hmm, foo() is an async function now,
   // and bar() needs the actual value, not the promise
   val |> await foo |> bar // error!
   // Oh yeah, need to transform "foo" into a lambda first...
   val |> (x=>await foo(x)) |> bar // error!
   // Oh dang, need to make that an async arrow function, stupid JS
   val |> (async x=>await foo(x)) |> bar // error! 
   // Argh, bar is still getting a promise!
   val |> (async x=>await foo(x)) |> (async x=>bar(await x)) // maybe?
   // Ugh, the pipeline still produces a promise at the end, stupid JS
   

   With luck, they recognize that this is a case handled by the special async form:

   val |> foo |> bar
   // hmm, foo() is an async function now,
   // and bar() needs the actual value, not the promise
   val |> await foo |> bar // error!
   // Oh yeah, awaits are special...
   val |> foo |> await |> bar