Context blocks for JavaScript

es-context-blocks.md

Context blocks for JavaScript

Warning: This is super unfinished. It won’t be finished for months.

I still think macros could work for JS. How many build tools could be collapsed into one? How many TC39 proposals could just go away if you could import new operators or control flow constructs from npm?

― tweet by Dave Herman

Motivation

Many programs contain the same repetitive contexts, containers, or computations. These include:

Context	Description	Examples
Promises and futures	Single-valued asynchronous computations	ES promises
Array/list comprehensions	Multi-valued eager sync computations	ES arrays (with forEach)
Sync iterators and generators	Multi-valued lazy sync computations	ES generators
Async iterators and generators	Multi-valued pulled async computations	ES async generators
Observables and event emitters	Multi-valued pushed async computations	RxJS, HTML EventEmitter
Server middleware	Async computations from requests to responses	Express.js middleware
Parsers and compilers	Sequential computations on strings	ANTLR, Yacc, GNU Bison, Parsec
Remote data sources	Computations on async-batched datasets	Haxl
Remote objects	Computations on async-batched properties	Cap’n Proto

These contexts all require specialized control flow, but that special control flow frequently mixes essential program details with repetitive boilerplate. This boilerplate becomes especially difficult to manage when we need to combine contexts together and process their values in complex ways. And, oftentimes, their special control flow requires us to use continuation-passing style, resulting in deeply nested callbacks (pyramids of doom). Specific examples are given in the following subsections.

Motivation: Promises

Promises are objects that represent the eventual completion (or failure) of some asynchronous operation. They are ubiquitous in JavaScript APIs.

Promises have specialized control flow: they have a “happy path” in which they resolve to their final values, and they have an alternative path in which they reject with an error.

The most basic way to combine promises is to pass callbacks to the promises’ methods:

fnA().then(a =>
  fnB(a).then(b =>
    fnC(a, b).then(c =>
      fnD(a, b, c),
    ),
  ),
).catch(err =>
  fnE(err),
);

Unfortunately, these deeply nested callbacks form pyramids of doom. (Chaining then calls is not possible because each consecutive step is dependent on all previous steps, rather than only the immediately previous step.)

The JavaScript language has specialized control-flow syntax for combining promises: async functions and await. Here, we use an async IIFE.

const combinedPromise = (async () => {
  try {
    const a = await fnA();
    const b = await fnB(a);
    const c = await fnC(a, b);
    return await fnD(a, b, c);
  } catch (err) {
    fnE(err);
  }
})();

These flatten the pyramid of doom and, they also replace repetitive boilerplate. We can follow the flow of data more easily.

This async/await syntax has a single purpose: combining single-valued asynchronous computations without deeply nested pyramids of doom. However, there are many other kinds of computations that would benefit from similar syntax.

Motivation: Sync iterators and generators

Iterators and generators are objects that represent a sequence of lazily evaluated values.

Motivation: Observables and event emitters

Observables represent one of the fundamental protocols for processing asynchronous streams of data. They are particularly effective at modeling streams of data such as user-interface events, which originate from the environment and are pushed into the application.

listen(element, 'keydown')
  .map(event => keyCommandMap.get(event.keyCode))
  .filter(keyCode => keyCode)
  .forEach(console.log);

…

const consolidatedData = forkJoin({
  foo: Observable.of(1, 2, 3, 4),
  bar: Promise.resolve(8),
  baz: timer(4000),
}).map(({ foo, bar, baz }) =>
  `${foo}.${bar}.${baz}`,
).forEach(console.log);
// Logs '4.8.0'.

Motivation: Parsers

Motivation: Remote data sources

Haxl is a Haskell library that simplifies access to remote “data sources”, such as databases, web-based services, document-management systems, compilers and linkers, or any other kind of API for fetching remote data. It automatically batches multiple requests to the same data source into a single request. It can request data from multiple data sources concurrently, and it caches previous requests. “Having all this handled for you means that your data-fetching code can be much cleaner and clearer than it would otherwise be if it had to worry about optimizing data-fetching” (Facebook (2014)).

The following Haskell code works in three stages: first the friends of id are fetched, and then all of the friends of those are fetched (concurrently), and finally those lists are concatenated to form the result.

friendsOfFriends id = do
  friends <- friendsOf id
  fofs <- mapM friendsOf friends
  return (concat fofs)

A JavaScript translation of this API has to use nested callbacks. It cannot use promises because the Haxl system must be able to see…

function getFriendsOfFriends (id) {
  return Haxl.applyFrom(friendsOf(id), friends =>
    Haxl.applyFrom(Haxl.map(friendsOf, friends), fofs =>
      Haxl.result(concat(fofs)));
  );
}

…

numCommonFriends xSource ySource = do
  xFriends <- friendsOf xSource
  yFriends <- friendsOf ySource
  return (length (intersect xFriends yFriends))

…

function getNumCommonFriends (xSource, ySource) {
  return Haxl.all([ friendsOf(xSource), friendsOf(ySource) ])
    .applyFrom(([ xFriends, yFriends ]) =>
      length(
        intersect(xFriends, yFriends),
      ),
    );
}

Unified solution for recurring patterns

JavaScript has improved the situation specifically for two types of computations: iterative generation and sequential async processing by adding specialized syntax for iterators / generator functions and for promises / async functions. However, these specialized syntaxes cannot be extended or customized, and they do not address the control flow of other contexts, containers, or computations.

Context blocks would be a unified, extensible system for abstracting away these other repetitive computations. With context blocks, context-sensitive computations are isolated to their blocks, with little disruption to the rest of the language and to the ecosystem. It can be easy to leak these context-sensitive computations outside of their contexts without abstractions that prevent us from doing so.

Context blocks would essentially form a lightweight macro system based on callbacks. Within any context block, a developer could extend the behavior of various control constructs, including =, await, yield, return, throw, try, and for, by overriding their behavior with a modifier object. Within a modified block, statements become calls to its modifier object’s methods, and each subsequent statement is nested in a callback to its previous statement’s method.

The modifier object determines which control constructs are valid. This metaprogramming would be powerful enough to explain the current async function and function * constructs (by using @Promise function and @Iterator function).
But it would also open the door to custom control-flow constructs from domain-specific languages, such as:
the Observables of RxJS,
the parsers of nearly.js or Parsimmon,
the queries of Microsoft’s LINQ,
or the data sources of Facebook’s Haxl,
…directly in JavaScript code, without dynamically parsed template literals or deeply nested callback pyramids of doom.

Railway-oriented programming

Many of these contexts share a common pattern: their computations generally runs on a “happy path”, but there is also often an alternative track that needs to be handled differently (such as expected operational exceptions). This railway-oriented programming…

Relationship with do expressions

Context blocks are similar to do expressions in that both embed a block of statements into an expression, but they differ in an important way.

Context blocks create and execute immediately invoked functions (IIFEs), while do expressions execute statements that are scoped to its surrounding function context. This means that, within a context block, await and yield affect only the context block. In contrast, within a do expression, await and yield affect the outer function context. (The in keyword is used, instead of the do keyword, in order to prevent confusion between these two behaviors.)

Prior art

Context blocks are based on F# computation expressions, which in turn was inspired by Haskell do notation and similar productions from other functional programming languages.

Type-theory details

Context blocks – and the contexts, containers, and computations they act on – are, in actuality, the infamous monads from functional programming (as well as their relatives applicative functors, and monoid transformers). Context blocks combine all of these abstract computation patterns into a unified comprehension syntax.

Monads, applicative functors, and monoids are recurring patterns from mathematics that appear in many functional programming languages, and which savvy developers may combine in many ways. This essay tries to avoid dwelling on their abstract theory, in favor of focusing on concrete examples of context blocks.

Nevertheless, the pattern is useful to simplify real-world code, by separating pipelines’ essential details from boilerplate logic (such as checking for failures or explicitly passing state data) at every step of each pipeline.

Scratchpad

Lots of old stuff

Context blocks

A context block is an expression, made of a block of statements with a modifier object.

Context blocks: Syntax

in @«modifier» { «bodyStatements» }

«modifier»

The context block’s modifier object. This expression has the same syntax as decorator expressions: a variable chained with property access (. only, no []) and calls (). To use an arbitrary expression as a modifier object, use @(expression).

A context block may be modified by only one modifier object.

«bodyStatements»

The statements that comprise the body of the context block. The modifier object determines which statement types are permitted.

Context blocks: Description

Contexts contain values

An context block executes (or will execute) its statements in a new context defined by the block’s modifier. The context block itself evaluates to a new context object.

The word “context” can apply to many different kinds of container objects and computation objects. Examples of context objects that are built into JavaScript include arrays, iterators, promises, and async iterators. Examples of context objects that come from libraries outside of the language include data streams, server middleware functions, and parser functions.

All context objects contain zero, one, or more values, and they might also throw errors. For example:

• An array contains zero or more values.
• An iterator contains zero or more (lazily evaluated) values,
  and it might throw an error in the middle of its execution.
• A promise (eventually) contains one value,
  and it might reject with an error before resolving its one value.
• An async iterator (eventually) contains zero or more (lazily evaluated) values,
  and it might reject with an error in the middle of its execution.

Operations inside context blocks

Inside a context block, the following operations and statements change behavior, depending on the modifier object’s methods. (If an operation or method requires a method that the modifier object does not implement, then a TypeError is thrown.)

Operation	Equivalent method	Meaning
v =* input	modifier.applyFrom( input, body)	Executes an input context and assigns input context’s resulting value(s) to a variable v inside the context block.
v0 =* input0 and v1 =* input1 and v2 =* input2	modifier.applyFrom( modifier.all( input0, input1, input2), body)	Executes an input0 context, an input1 context, and an input2 context possibly in parallel, and assigns the resulting value(s) of each to variables v0, v1, and v2 inside the context block. The input0 context, input1 context, and input2 context expressions cannot depend on one another; they can only depend on variables declared beforehand. In other words, v0 is not in scope to input1 context and input2 context.
void* input	modifier.applyFrom( input, body)	Executes an input context and throws away its value(s). This is equivalent to: _ =* input …where _ is a dummy variable that is never used in the rest of the context block.
return value;	modifier.return( value)	Returns a single value from inside the context block. That value becomes the value (or one of the values) that the context block’s final context object contains.
return* input;	modifier.returnFrom( input)	Returns an input context’s value(s), if any, from inside the context block. Those value(s) become the value (or one of the values) that the context block’s final context object contains.
yield value	modifier.yield( value)	Yields a value from inside the context block. That value becomes the value (or one of the values) that the context block’s final context object contains. (A modifier object should implement yield only when it would make sense: when its contexts may contain multiple values.)
yield* input	modifier.yieldFrom( input)	Yields an input context’s value(s) from inside the context block. That value becomes the value (or one of the values) that the context block’s final context object contains. (A modifier object should implement yield* only when it would make sense: when its contexts may contain multiple values.)

context blocks: Examples

Context block	Equivalent callbacks
const numCommonFriends = in @Haxl { const xFriends =* friendsOf(x) and yFriends =* friendsOf(y); return length( intersect(xFriends, yFriends), ); }	const numCommonFriends = Haxl.run(() => Haxl.delay(() => Haxl.applyFrom( Haxl.all(friendsOf(x), friendsOf(y)), ([ xFriends, yFriends ]) => Haxl.return( length( intersect(xFriends, yFriends), )))));
const query = in @LINQ { for ( const student of db.Student and selection of db.CourseSelection ) { if (student.studentID === selection.studentID) yield { student, selection }; } }	const query = LINQ.run(() => LINQ.delay(() => LINQ.for( LINQ.all(db.Student, db.CourseSelection), ([ student, selection ]) => { if (student.studentID === selection.studentID) { return LINQ.yield({ student, selection }); } else { return LINQ.zero(); } })));
const items = in @Iterator { console.log('Starting generator'); yield 'start'; yield* getItems(); yield 'end'; console.log('Ending generator'); }	const items = Iterator.run(() => Iterator.delay(() => Iterator.combine( ( console.log('Starting generator'), Iterator.zero()), Iterator.delay(() => Iterator.combine( Iterator.yield('start'), Iterator.delay(() => Iterator.combine( Iterator.yieldFrom(getItems()), Iterator.combine( Iterator.delay(() => Iterator.yield('end'), Iterator.delay(() => ( console.log('Ending generator'), Iterator.zero())))))))))));
const jsonPromise = in @Promise { console.log('Fetching…'); const response = await fetch(url); try { const json = await response.json(); console.log(json); return json; } catch (err) { console.error(err); throw new Error('Invalid JSON'); } }	const jsonPromise = Promise.run(() => Promise.delay(() => ( console.log('Fetching…'), Promise.zero()), Promise.await(fetch(url), response => Promise.tryCatch( Promise.delay(() => Promise.await( response.json(), json => Promise.delay(() => ( console.log(json), Promise.return(json))))), err => ( console.log(err), return Promise.throw('Invalid JSON'))))));

=* (assign-from)

=*: Syntax

=*: Description

modifier.applyFrom somehow extracts zero or more values from input context and then, for each value, calls body(value).

(If modifier.delay is implemented, then the remainder of the context block is delayed until modifier.applyFrom calls body(value). If modifier.applyFrom returns without ever calling body, then the context block short-circuits and terminates without continuing on after this expression.)

---

modifier.applyFrom somehow extracts zero or more value0s from input0 context, zero or more value1s from input1 context, and zero or more values2s from input2 context. Then, for each value0, value1, and value2 it calls body([ value0, value1, and value2 ])**.

do* (do-from)

(Just like with =*, if modifier.delay is also implemented, then do* can short-circuit and terminate the context block, depending on whether modifier.applyFrom, based on input context, calls its body at least once.)

return

return*

yield

yield*

await

for of

try catch finally

Context block functions

An context function is an expression or function declaration that modifies its body with a modifier object. The in…function form can be used to create a context function.

Context functions: Syntax

@«modifier» function «name» («parameters») { «bodyStatements» }

«name»

The name of the context function. This is optional if this is in a function expression, and it is required if this is in a function declaration.

«modifier»

The context function’s modifier object. This expression has the same syntax as decorator expressions: a variable chained with property access (. only, no []) and calls (). To use an arbitrary expression as a modifier object, use @(expression).

A context function may have only one modifier object.

«parameters»

A comma-separated list of the context function’s parameters.

«bodyStatements»

The statements that comprise the body of the context function. The modifier object determines which statement types are permitted.

Context block functions: Description

Extensions

Context block blocks extend the meaning of several existing syntax constructs.

The * suffix in yield is generalized to mean “from” another context, as in “yield from”, “return from”, or “assign from”. Formerly it meant only “yield from another generator”.

Current syntax	Extended meaning in a context block
yield value yields a value from inside any gen. or async gen.	yield value yields a value from inside any context block with a yield method.
yield* input yields an input iterator’s value(s) from inside any gen. or async gen., possibly delaying evaluation while waiting for input.	yield* input yields an input context’s value(s) from inside any context block with a yieldFrom method, possibly delaying evaluation while waiting for input.
return value returns a value from inside any fn., gen., async fn., or async gen.	return value returns a value from inside any context block with a return method.
	return* input returns an input context’s value(s) from inside any context block with a returnFrom method, possibly delaying evaluation while waiting for input.
const v = value applies a value’s value to a variable inside any fn., gen., async fn., or async gen.	const v = value applies a value’s value to a variable inside any context block.
	const v =* input applies an input context’s value(s) to a variable inside any context block with a apply method, possibly delaying evaluation while waiting for input.
await input evaluates to an input promise’s value inside any async fn. or async gen, possibly delaying evaluation while waiting for input.	await input evaluates to an input context’s value inside any context block with an await method, possibly delaying evaluation while waiting for input.
try { ts } catch (err) { cs } first evaluates ts and, whenever ts throws an error err (inside any outer fn., gen., async fn., or async gen.) or rejects with an error err (inside any outer async fn. or async gen.), applies err to a variable and evaluates cs.	try { ts } catch (err) { cs } first evaluates ts and, whenever any of them throws or rejects with an error err (inside any context block with a tryCatch method), applies err to a variable and evaluates cs.
for (const value of input) iterates over an input iterator, applying each value to a variable inside any fn., gen., async fn., or async gen., possibly delaying evaluation while waiting for input.	for (const value of input) iterates over an input context, applying each value to a variable inside any context block with a for method, possibly delaying evaluation while waiting for input.

Status quo	With context block
Promise.all([ (async () => { const result = await fetch('thing A'); return result.json(); })(), (async () => { const result = await fetch('thing B'); return result.json(); })(), ]).then(([a, b]) => `${a}, ${b}`)); // Creates a Promise that will resolve to // 'thing A, thing B'.	in @Promise { const a =* in @Promise { return fetch('thing A'); } and b =* in @Promise { return fetch('thing B'); }; return `${a}, ${b}`; }; // Creates a Promise that will resolve to // 'thing A, thing B'.
const observable = forkJoin({ foo: rx.of(1, 2, 3, 4), bar: Promise.resolve(8), baz: timer(4000), }).pipe( map(({ foo, bar, baz }) => `${foo}.${bar}.${baz}`)); // Creates an Observable that emits '4.8.0'.	@rx.Observable in { const foo =* rx.of(1, 2, 3, 4) and bar =* Promise.resolve(8) and baz =* timer(4000); yield `${foo}.${bar}.${baz}`; }; // Creates an Observable that emits '4.8.0'.
const weight = rx.of(70, 72, 76, 79, 75); const height = rx.of(1.76, 1.77, 1.78); const bmi = rx.combineLatest([weight, height]).pipe( map(([w, h]) => w / (h* h)), map(bmi => `BMI is ${bmi}`), ); // Creates an Observable that emits: // 'BMI is 24.212293388429753', // 'BMI is 23.93948099205209', // 'BMI is 23.671253629592222'.	const combined = @rx.Observable in { for ( const w of rx.of(70, 72, 76, 79, 75) and h of rx.of(1.76, 1.77, 1.78) ) { const bmi = const w / (h* h); yield `BMI is ${bmi}`; } }; // Creates an Observable that emits: // 'BMI is 24.212293388429753', // 'BMI is 23.93948099205209', // 'BMI is 23.671253629592222'.
expression -> number "+" number {% function (data) { return { operator: 'sum', leftOperand: data[0], rightOperand: data[2], }; } %}	const expressionRule = @n.Rule in { const leftOperand =* numberRule and _ =* n.term('+') and rightOperand =* numberRule; return { operator: 'sum', leftOperand, rightOperand, }; };
const monthRangeParser = P.seq( monthParser, P.string('-'), monthParser, ).map(([ firstMonth, _, secondMonth ]) => [ firstMonth, secondMonth ], );	const monthRangeParser = @P.Parser in { const firstMonth =* monthRangeParser and _ =* P.string('-') and secondMonth =* $number; return [ firstMonth, secondMonth ]; };
function cookieParser (req, res, next) { const cookies = c.parseCookies(req); req.cookies = cookies; next(); } function cookieValidator (req, res, next) { if (validateCookies(req.cookies)) next(); else next(new SyntaxError('invalid cookies')); } function errorHandler (err, req, res, next) { res.send(`Error: {err}`); } async function cleaner (req, res, next) { await cleanUp(req, res); next(); } const router = e.Router(); router.get('/', cookieParser, cookieValidator, (req, res, next) => { const { cookies } = req; const cookieStr = JSON.stringify(cookies); res.send(`Cookies: ${cookieStr}`); next(); }, cleaner, errorHandler, (err, req, res, next) => cleaner(req, res, next));	function cookieParser (req, res, next) { const cookies = e.parseCookies(req); req.cookies = cookies; next(); } function cookieValidator (req, res, next) { if (validateCookies(req.cookies)) next(); else next(new SyntaxError('invalid cookies')); } function createErrorHandler (err) { return (req, res, next) => { res.send(`Error: {err}`); }; } async function cleaner (req, res, next) { await cleanUp(req, res); next(); } const router = e.Router(); router.get('/', @e.Middleware in { do* cookieParser; try { do* cookieValidator and* (req, res, next) => { const { cookies } = req; const cookieStr = JSON.stringify(cookies); res.send(`Cookies: ${cookieStr}`); next(); }; } catch (err) { do* createErrorHandler(err); } finally { do* cleaner; } });

F# for Fun and Profit

Introduction: Unwrapping the enigma...

function log (p) {
  console.log(`expression is ${p}`);
}

// Prints:
// expression is 42
// expression is 43
// expression is 85
const loggedWorkflow = in {
  const x = 42;
  log(x);
  const y = 43;
  log(y);
  const z = x + y;
  log(z);
  return z;
}

const logging = {
  apply (input, body) {
    log(input);
    return body(input);
  },
  
  return (value) {
    return value;
  },
};

// Prints:
// expression is 42
// expression is 43
// expression is 85
in @logging {
  const x =* 42;
  const y =* 43;
  const z =* x + y;
  return z;
};

Safe division

function safelyDivide (top, bottom) {
  if (bottom == 0)
    return null;
  else
    return top / bottom;
}

function safelyDivideThrice (top, bottom0, bottom1, bottom2) {
  return in {
    const quotient0 = safelyDivide(top, bottom0);
    if (quotient0 == null) {
      return quotient0;
    } else {
      const quotient1 = safelyDivide(quotient0, bottom1);
      if (quotient1 == null) {
        return quotient1;
      } else {
        return safelyDivide(quotient1, bottom2);
      }
    }
  };
}
safelyDivideThrice(12, 3, 2, 1); // Returns 2
safelyDivideThrice(12, 3, 0, 1); // Returns 0

const maybe = {
  apply (input, body) {
    if (input == null)
      return input;
    else
      return body(input);
  },
  
  return (value) {
    return value;
  },
};

function safelyDivideThrice (top, bottom0, bottom1, bottom2) {
  return in @maybe {
    const quotient0 =* safelyDivide(top, bottom0);
    const quotient1 =* safelyDivide(quotient0, bottom1);
    const quotient2 =* safelyDivide(quotient1, bottom2);
    return quotient2;
  };
}
safelyDivideThrice(12, 3, 2, 1); // Returns 2
safelyDivideThrice(12, 3, 0, 1); // Returns 0

Chains of or-else tests

const obj0 = { '1': 'One', '2': 'Two' };
const obj1 = { A: 'Alice', B: 'Bob' };
const obj2 = { CA: 'California', NY: 'New York' };
function multilookup (key) {
  const val0 = map0.get(key);
  if (val0 != null) {
    return val0;
  } else {
    const val1 = map1.get(key);
    if (val1 != null) {
      return val1;
    } else {
      return map2.get(key);
    }
  }
}
multilookup('A'); // Returns 'Alice'
multilookup('CA'); // Returns 'California'
multilookup('X'); // Returns undefined

const orElse = {
  returnFrom (input) {
    return input;
  },
  
  combine (input0, input1) {
    if (input0 != null)
      return input0;
    else
      return input1;
  },
  
  delay (body) {
    return body();
  },
};

const obj0 = { '1': 'One', '2': 'Two' };
const obj1 = { A: 'Alice', B: 'Bob' };
const obj2 = { CA: 'California', NY: 'New York' };
function multilookup (key) {
  return in @orElse {
    return* map0.get(key);
    return* map1.get(key);
    return* map2.get(key);
  };
}
multilookup('A'); // Returns 'Alice'
multilookup('CA'); // Returns 'California'
multilookup('X'); // Returns undefined

Understanding continuations: How let works behind the scenes

function divideWithCallback (top, bottom, callback) {
  return in {
    if (bottom == 0) {
      return callback('divideBy0');
    } else {
      const quotient = top / bottom;
      return callback(null, quotient);
    }
  };
}
divideWithCallback(6, 3, logResult); // Prints: Good input. 2
divideWithCallback(6, 0, logResult); // Prints: Bad input! divideBy0

The “logging” example revisited

const logging = {
  apply (input, body) {
    log(input);
    return body(input);
  },
  
  return (value) {
    return value;
  },
};

// Prints:
// expression is 42
// expression is 43
// expression is 85
in @logging {
  const x =* 42;
  const y =* 43;
  const z =* x + y;
  return z;
}

// Prints:
// expression is 42
// expression is 43
// expression is 85
logging.applyFrom(42, x =>
  logging.applyFrom(43, y =>
    logging.applyFrom(x + y), z =>
      logging.return(z)));

The “safe divide” example revisited

function logResult (err, value) {
  if (err) {
    console.error(`Bad input! ${err}`);
  } else {
    console.log(`Good input. ${value}`)
  }
}

const maybe = {
  apply (input, body) {
    if (input == null)
      return input;
    else
      return body(input);
  },
  
  return (value) {
    return value;
  },
};

function safelyDivideThrice (top, bottom0, bottom1, bottom2) {
  return in @maybe {
    const quotient0 =* safelyDivide(top, bottom0);
    const quotient1 =* safelyDivide(quotient0, bottom1);
    const quotient2 =* safelyDivide(quotient1, bottom2);
    return quotient2;
  };
}
safelyDivideThrice(12, 3, 2, 1); // Returns 2
safelyDivideThrice(12, 3, 0, 1); // Returns 0

function safelyDivideThrice (top, bottom0, bottom1, bottom2) {
  return maybe.applyFrom(safelyDivide(top, bottom0), quotient0 =>
    maybe.applyFrom(safelyDivide(quotient0, bottom1), quotient1 =>
      maybe.return(safelyDivide(quotient1, bottom2))));
}
safelyDivideThrice(12, 3, 2, 1); // Returns 2
safelyDivideThrice(12, 3, 0, 1); // Returns 0

Introducing apply: Steps towards creating our own let

let leftHandSide =* rightHandSide; body;

context.applyFrom(rightHandSide, leftHandSide => body);

Context block and wrapper types: Using types to assist the workflow

const result = in @maybe {
  const int0 =* expression0;
  const int1 =* expression1;
  return int0 + int1;
};

function Success (value) {
  return { type: 'success', value };
}
function Failure (err) {
  return { type: 'failure', err };
}
function getCustomer (name) {
  if (name)
    return Success('Cust42');
  else
    return Failure('getCustomerID failed');
}
function getLastOrderForCustomer (customerID) {
  if (customerID)
    return Success('Order123');
  else
    return Failure('getLastOrderForCustomer failed');
}
function getLastProductForOrder (orderID) {
  if (orderID)
    return Success('Product456');
  else
    return Failure('getLastProductForOrder failed');
}

const product = in {
  const result0 = getCustomer('Alice');
  switch (result0.type) {
    case 'failure':
      return result0;
    case 'success': {
      const customerID = result0.value;
      const result1 = getLastOrderForCustomer(customerID);
      switch (result1.type) {
        case 'failure':
          return result1;
        case 'success': {
          const orderID = getLastOrderForCustomer(result1.value);
          const result2 = getLastProductForOrder(orderID);
          switch (result2.type) {
            case 'failure':
              return result2;
            case 'success': {
              const productID = result2.value;
              console.log(`Product is ${productID}`);
            }
          }
        }
      }
    };
  }
};

const DatabaseResult = {
  apply (input, body) {
    switch (input.type) {
      case 'failure':
        return input;
      case 'success':
        return body(input);
    }
  },
  
  return (value) {
    return Success(value);
  }
};

const product = in @DatabaseResult {
  const customerID =* getCustomer('Alice');
  const orderID =* getLastOrderForCustomer(customerID);
  const productID =* getLastProductForOrder(orderID);
  console.log(`Product is ${productID}`);
};

const maybe = {
  apply (input, body) {
    if (input == null)
      return input;
    else
      return body(input);
  },
  
  return (value) {
    return value;
  },
  
  returnFrom (input) {
    return input;
  },
};

in @maybe {
  return 1;
};

in @maybe {
  return* 1;
};

More on wrapper types: We discover that even lists can be wrapper types

Can non-generic wrapper types work?

const StringInt = {
  apply (input, body) {
    let value;
    try {
      value = parseInt(input);
    } catch (err) {
      return err;
    }
    return body(value);
  },
  
  return (value) {
    return value.toString();
  }
};

const good = in @StringInt {
  const i =* '42';
  const j =* '43';
  return i + j;
};
// good is now '85'

const bad = in @StringInt {
  const i =* '42';
  const j =* 'xxx';
  return i + j;
};
// bad is now a SyntaxError about 'xxx'

in @StringInt {
  const i =* '99';
  return i;
}; // This evaluates to '99'

in @StringInt {
  const i =* 'xxx';
  return i;
}; // This evaluates to a SyntaxError

Rules for workflows that use wrapper types

const output0 = in @modifier {
  const container = input;
  const value =* container;
  return value;
}
const output1 = in @modifier {
  const container = input;
  return* container;
}
// output0 and output1 should be equivalent.

in @modifier {
  const container0 = input;
  const container1 = in @modifier {
    const value =* container0;
    return value;
  };
  // container0 and container1 should be equivalent.
};

const output0 = in @modifier {
  const x =* input;
  const y =* f(x);
  return* g(y);
};
const output1 = in @modifier {
  const y =* in @modifier {
    const x =* input;
    return* f(x);
  };
  return* g(y);
};
// output0 and output1 should be equivalent.

Lists as wrapper types

const each = {
 * apply (input, body) {
    for (const value of input)
      yield body(value);
  },
  
 * return (value) {
    yield value;
  },
};

const sums = Array.from(in {
  const i =* [ 0, 1, 2 ];
  const j =* [ 10, 11, 12 ];
  return i + j;
});
// [ 10, 11, 12, 11, 12, 13, 12, 13, 14 ]

const products = Array.from(in {
  const i =* [ 0, 1, 2 ];
  const j =* [ 10, 11, 12 ];
  return i* j;
});
// [ 0, 10, 20, 0, 11, 22, 0, 12, 24 ]

Syntactic sugar for “for”

const each = {
 * apply (input, body) {
    for (const value of input)
      yield body(value);
  },
  
 * return (value) {
    yield value;
  },
  
 * for (input, body) {
    return this.applyFrom(input, body);
  },
};

const products = Array.from(in {
  for (const i of [ 0, 1, 2 ])
    for (const j of [ 10, 11, 12 ])
      return i * j;
});
// [ 0, 10, 20, 0, 11, 22, 0, 12, 24 ]

The identity “wrapper type”

const identity = {
  apply (input, body) {
    return body(input);
  },
  
  return (value) {
    return value;
  },
  
  returnFrom (value) {
    return value;
  },
};

Implementing a builder: zero and yield, getting started with the basic builder methods

Wrapped and unwrapped types

in @maybe {
  const [ x, y ] =* [ 0, 1 ];
  const [ first, ...rest ] =* [ 0, 1, 2 ];
};

in @maybe {
  const [ x, y ] =* [ 0, 1 ];
  const [ first, ...rest ] =* [ 0, 1, 2 ];
};

Implementing special methods in the builder class (or not)

in @maybe {
  for (const i of [ 0, 1, 2 ])
    return i;
};
// A context block may use a `for` statement
// only when its modifier object defines a “for” method.

in @maybe {
  1;
};
// A context block may implicitly return
// only if its modifier object defines a “zero” method.

Operations with and without ‘*’

const value = 1; // Right-hand side is a value.
const value =* [ 1 ]; // Right-hand side is a container.
return 1; // Right-hand side is a value.
return* [ 1 ]; // Right-hand side is a container.
yield 1; // Right-hand side is a value.
yield* [ 1 ]; // Right-hand side is a container.

Diving in: creating a minimal implementation of a workflow

const tracing = {
  apply (input, body) {
    const { err, value } = input;
    if (err != null) {
      console.log('Tried to apply error input. Exiting.')
      return input;
    } else {
      console.log(`applying value ${value}. Continuing.`);
      return body(value);
    }
  },
  
  return (value) {
    console.log(`Returning value ${value}.`);
    return { value };
  },
  
  returnFrom (input) {
    console.log(`Returning input ${input} directly.`);
    return input;
  },
};

in @tracing {
  return 1;
};
in @tracing {
  return* { value: 1 };
};
in @tracing {
  const x =* { value: 1 };
  const y =* { value: 2 };
  return x + y;
};
in @tracing {
  const x =* { value: 1 };
  const y =* { error: 1 };
  return x + y;
};

Introducing “do *”

in @tracing {
  do* { value: 1 };
  do* { value: 1 };
  const x =* { value: 1 };
  return x;
};

Introducing “Zero”

in @tracing {};
// A context block may implicitly return
// only if its modifier object defines a “zero” method.

in @tracing {
  console.log('Hello');
};
in @tracing {
  if (false)
    return 1;
};
// A context block may implicitly return
// only if its modifier object defines a “zero” method.

const tracing = {
  // Other methods as before.

  zero () {
    console.log(`Returning zero.`);
    return { error: 1 };
  },
};

in @tracing {
  console.log('Hello');
};
// Prints 'Hello' then 'Returning zero.'.
in @tracing {
  if (false)
    return 1;
};
// Prints 'Returning zero.'.

Introducing “Yield”

in @tracing {
  yield 1;
};
// A context block may yield
// only if its context object defines a “yield” method.

const tracing = {
  // Other methods as before.

  yield (value) {
    console.log(`Yielding value ${value}.`);
    return { value };
  },

  yieldFrom (input) {
    console.log(`Yielding input ${input} directly.`);
    return input;
  },
};

in @Iterator { yield 1; }; // OK
in @Iterator { return 1; }; // Error

in @Promise { yield 1; }; // Error
in @Iterator { return 1; }; // OK

Revisiting “For”

const each = {
 * apply (input, body) {
    for (const value of input)
      yield body(value);
  },
  
 * return (value) {
    yield value;
  },

 * yield (value) {
    yield value;
  },

 * for (input, body) {
    return this.applyFrom(input, body);
  },
};

in @Iterator {
  const x =* [ 0, 1, 2 ];
  const y =* [ 10, 11, 12 ];
  yield x + y;
};

in @Iterator {
  for (const x of [ 0, 1, 2 ])
    for (const y of [ 10, 11, 12 ])
      yield x + y;
};

Implementing a builder’s Combine: How to return multiple values at once

in @tracing {
  yield 1;
  yield 2;
};
in @tracing {
  return 1;
  return 2;
};
in @tracing {
  if (true) console.log('hello');
  return 1;
};
// A context block may combine statements
// only if its context object defines a “combine” method and a “delay” method.

const tracing = {
  // Other methods as before.

  combine (input0, input1) {
    if (input0.err != null) {
      if (input1.err != null) {
        console.log(`Combining two errors ${input0} and ${input1}.`);
        return { err: input0.err + input1.err };
      } else {
        console.log(`Combining error ${input0} with non-error ${input1}.`);
        return input1;
      }
    } else {
      if (input1.err != null) {
        console.log(`Combining non-error ${input0} with error ${input1}.`);
        return input0;
      } else {
        console.log(`Combining two non-errors ${input0} and ${input1}`);
        return { value: input0.value + input1.value };
      }
    }
  },
  
  delay (body) {
    console.log('Delay.');
    return body();
  },
};

in @tracing {
  yield 1;
  yield 2;
};
// Delay. Yielding value 1. Delay. Yielding value 2. Combining two non-errors 1 and 2.

in @tracing {
  yield 1;
  return 2;
};
// Delay. Yielding value 1. Delay. Returning value 2. Combining two non-errors 1 and 2.

Using Combine for sequence generation

const each = {
 * apply (input, body) {
    for (const value of input)
      yield body(value);
  },

 * yield (value) {
    yield value;
  },

  yieldFrom (input) {
    return input;
  },

 * for (input, body) {
    return this.applyFrom(input, body);
  },
  
 * combine (input0, input1) {
    yield* input0;
    yield* input1;
  },
};

in @Iterator {
  yield 1;
  yield 2;
};

in @Iterator {
  yield 1;
  yield* [ 2, 3 ];
};

Array.from(in @Iterator {
  for (const i of [ 'red', 'blue' ]) {
    yield i;
    for (const j of [ 'hat', 'tie' ]) {
      yield* [ `${i} ${j}`, `-` ];
    }
  }
});
// ["red", "red hat", "-", "red tie", "-", "blue", "blue hat", "-", "blue tie", "-"]

in @Iterator {
  const x =* [ 0, 1, 2 ];
  const y =* [ 10, 11, 12 ];
  yield x + y;
};

in @Iterator {
  for (const x of [ 0, 1, 2 ])
    for (const y of [ 10, 11, 12 ])
      yield x + y;
};

Order of processing for “combine”

Array.from(in @Iterator {
  yield 1; yield 2; yield 3; yield 4;
});
Array.from(Iterator.combine(
  Iterator.yield(1),
  Iterator.combine(
    Iterator.yield(2),
    Iterator.combine(
      Iterator.yield(3),
      Iterator.yield(4)))));

Combine for non-sequences

in @modifier {
  if (true)
    console.log('hello');
  return 1;
};

const orElse = {
  zero () {
    return { type: 'failure' };
  },
  
  combine (input0, input1) {
    switch (input0.type) {
      case 'success':
        return input0;
      case 'failure':
        return input1;
      default:
        throw new TypeError;
    }
  },
};

Example: Parsing

function intParser (inputString) {
  try {
    const value = parseInt(inputString);
    return { type: 'success', value };
  } catch (err) {
    return { type: 'failure' };
  }
}

function boolParser (inputString) {
  switch (inputString) {
    case 'true':
      return { type: 'success', value: true };
    case 'false':
      return { type: 'success', value: false };
    default:
      return { type: 'failure' };
  }
}

in @orElse {
  return* boolParser('42');
  return* intParser('42');
}
// Results in { type: 'success': value: 42 }

Guidelines for mixing “Combine” and “Zero”

context.combine(input, context.zero());
context.combine(context.zero(), input);
input;

F# Context Expression Zoo

Containers

@maybe {
  if (b == 0)
    return* none;
  console.log('Calculating...');
  return a / b;
};

maybe.run(
  maybe.delay(() =>
    maybe.combine(
      in {
        if (b == 0)
          maybe.returnFrom(none),
        else
          maybe.zero(),
      },
      maybe.delay(() => {
        console.log('Calculating...');
        return maybe.return(a / b);
      }))));

Asynchronous workflows

const answerPromise = @Promise {
  console.log('Welcome...');
  42;
};

const answerPromise = Promise.run(
  Promise.delay(() => {
    console.log('Welcome...');
    Promise.return(42);
  }));

@Promise function getLength (url) {
  const html =* fetch(url);
  await sleep(1000);
  return html.length;
}

const lengthPromise = @Promise {
  const html = await fetch(url);
  await sleep(1000);
  html.length;
};

const lengthPromise = @Promise {
  const html =* fetch(url);
  void sleep(1000);
  html.length;
};

@Promise {
  if (delayFlag)
    await sleep(1000);
  console.log('Starting...');
  await fetch(url);
}

Promise.combine(
  in {
    if (delayFlag)
      Promise.applyFrom(sleep(1000), Promise.zero()),
    else
      Promise.zero();
  },
  Promise.delay(() => {
    console.log('Starting...');
    return Promise.returnFrom(fetch(url));
  }),
});

Parsers

@parser function atLeast1 ($input) {
  const firstValue =* $input;
  const restValues =* atLeast0($input);
  return [ firstValue, ...restValues ];
}

@parser function atLeast0 ($input) {
  return* atLeast1($input);
  return [];
}

function atLeast0 ($input) {
  return parser.combine(
    parser.returnFrom(atLeast1($input)),
    parser.delay(() => parser.return([])));
}

List comprehensions

@Iterator {
  for (const n of arr) {
    yield n;
    yield n* 10;
  }
};

Iterator.for(arr, n =>
  Iterator.combine(
    Iterator.yield(n),
    Iterator.delay(() => Iterator.yield(n* 10))));

Asynchronous sequences

@PromiseGenerator function getURLEverySecond (n) {
  await sleep(1000);
  yield getURL(n);
  yield* getURLEachSecond(n + 1);
}

@PromiseGenerator function getPageEverySecond (n) {
  for (url of getURLEverySecond(n)) {
    const html = await fetch(url);
    yield { url, html };
  }
}

Applicative formlets

@Form {
  const name =* textBox()
  and gender =* dropDown(genderArr);
  return `${name}: ${gender}`;
};

Form.map(
  Form.mergeSources(textBox(), dropDown()),
  (name, gender) => `${name}: ${gender}`);

Rethinking Applicatives in F#

in @Result {
  const a =* result0
  and b =* result1
  and c =* result2;
  return a + b - c;
};

// AWS DynamoDB reader
in @DynamoDB {
  const id =* guidAttrReader('id')
  and email =* stringAttrReader('email')
  and verified =* boolAttrReader('verified')
  and dob =* dateAttrReader('dob')
  and balance =* decimalAttrReader('balance');

  return { id, email, verified, dob, balance };
};

// Reads the values of x, y and z concurrently, then applies f to them.
in @Promise {
  const x =* slowRequestX()
  and y =* slowRequestY()
  and z =* slowRequestZ();
  return f(x, y, z);
}
// This is equivalent to:
// Reads the values of x, y and z concurrently, then applies f to them.
in @Promise {
  const [ x, y, z ] =
    await Promise.all([
      slowRequestX(),
      slowRequestY(),
      slowRequestZ(),
    ]);
  return f(x, y, z);
}

// One context expression gives both the behaviour of the form and its structure
const form = in @inForm {
  const name =* createTextBox()
  and gender =* createDropDown(genderList);

  return `${name} is ${gender}`;
}
const formID = 0;
const html = form.render(formID);
const result = form.evaluate(formID);

// Outputs a + b, which is recomputed every time foo or bar outputs a new value,
// avoiding any unnecessary resubscriptions
in @Observable {
  const valueA =* inputObservableA
  and valueB =* inputObservableB;

  return valueA + valueB;
};

// If both reading from the database or the file go wrong, the context
// can collect up the errors into a list to helpfully present to the user,
// rather than just immediately showing the first error and obscuring the
// second error
in @Result {
  const users =* readUsersFromDb()
  and birthdays =* readUserBirthdaysFromFile(filename);

  return updateBirthdays(users, birthdays);
}

// One context expression gives both the behaviour of the parser
// (terms in of how to parse each element of it, what their defaults should
// be, etc.) and the information needed to generate its help text
in @CLIOptions {
  const username =* option('username', '')
  and fullname =* option('fullname', undefined, '')
  and id =* option('id', undefined, parseInt);

  return new User(username, fullname, id)
}

Open-sourcing Haxl, a library for Haskell

@Haxl function getNumCommonFriends (x, y) {
  const xFriends =* friendsOf(x)
  and yFriends =* friendsOf(y);

  return length(intersect(xFriends, yFriends));
}

An example data source for accessing the Facebook Graph API

@Haxl function initializeGlobalState (threads, credentials, token) {
  const manager =* newManager(tlsManagerSettings)
  and semaphore =* newQSem(threads);

  return new FacebookState({ credentials, manager, userAccessToken, semaphore });
}

@Haxl function fetchFBRequest (token, userFriendsGetter) {
  const { id } = userFriendsGetter;
  const friends =* getUserFriends(id, [], token);
  const source =* fetchAllNextPages(friends);

  return consume(source);
}

There is no Fork: an Abstraction for Efficient, Concurrent, and Concise Data Access

@Haxl function renderMainPane () {
  const posts =* getAllPostsInfo();
  const orderedPosts = posts
    .sort((p0, p1) => compare(p0.postDate, p1.postDate))
    .take(5);
  const content =* orderedPosts
    .mapM(p => p.postID |> getPostContent(%));

  return* renderPosts(zip(orderedPosts, content));
}

@Haxl function renderPopularPosts () {
  const postIDs =* getPostIDs();
  const views =* postIDs.map(getPostViews);
  const orderedViews = zip(postIDs, views)
    .sort((p0, p1) => compare(p0.snd, p1.snd))
    .map(fst)
    .take(5);
  const content =* orderedViews.map(getPostDetails);

  return renderPostList(content);
}

Ron Buckton’s LINQ examples

Not sure if it helps or hurts my case, but here's an example of a PowerShell pipeline for posh-vsdev (a powershell module I wrote to help with switching between visual studio developer command lines):

Get-ChildItem -Path:"HKCU:\Software\Microsoft\VisualStudio\*.0" -PipelineVariable:ProductKey
    | ForEach-Object -PipelineVariable:ConfigKey {
        Join-Path -Path:$local:ProductKey.PSParentPath -ChildPath:($local:ProductKey.PSChildName + "_Config")
            | Get-Item -ErrorAction:SilentlyContinue;
    }
    | ForEach-Object -PipelineVariable:ProfileKey {
        Join-Path -Path:$local:ProductKey.PSPath -ChildPath:"Profile"
            | Get-Item -ErrorAction:SilentlyContinue
    }
    | ForEach-Object {
        $local:Version = $local:ProfileKey | Get-ItemPropertyValue -Name BuildNum;
        $local:Path = $local:ConfigKey | Get-ItemPropertyValue -Name ShellFolder;
        $local:Name = "VisualStudio/$local:Version";
        if (Join-Path -Path:$local:Path -ChildPath:$script:VSDEVCMD_PATH | Test-Path) {
            [VisualStudioInstance]::new(
                $local:Name,
                "Release",
                $local:Version -as [version],
                $local:Path,
                $null
            );
        }
    };
}

I needed to use -PipelineVariable to be able to rename the topic to use it in nested pipelines.

getChildItem("HKCU...")
|> (productKey => 
  forEachObject(...)
  |> (configKey => 
    forEachObject(...)
    |> (profileKey => 
       forEachObject(productKey, configKey, profileKey)
)))

Powershell pipes values into a function, which can handle things in two ways: either as an -InputObject argument, or by using a begin/end/process body. The {} are script blocks that can take explicit and implicit (i.e., topic) parameters.

in @forEachObject {
  const productKey =* getChildItem("HKCU…");
  const * configKey =* …;
  const * profileKey =* …;
  something(productKey, configKey, profileKey);
}

https://github.com/rbuckton/iterable-query-linq

Uses LINQ's [FLWOR]-like syntax for comprehensions, with the downside of needing to parse the template tag to produce output. :/

We've been discussing typed template arrays for TS, and with template literal types I might actually be able to make a subset of the dialect typesafe, heh.

typed template arrays meaning:

declare function tag<T extends TemplateStringsArray, A extends any[]>(array: T, ...args: A): [T, A];

from`abc${1}def`; // type: [["abc", "def"] & { raw: ["abc", "def"] }, [1]]

Use conditional types and template literal types to parse the generic template strings array and inject types from the tuple... Feasible but ugly.

With the template syntax in the repo above, I'd write it like this:

linq`
  from productKey of ${getChildItem("HKCU...")}
  let joinPath = ${joinPath} 
  let getItem = ${getItem}
  let getItemPropertyValue = ${getItemPropertyValue}
  let testPath = ${testPath}
  let vsDevCmdPath = ${vsDevCmdPath}
  from configKey of getItem(joinPath(productKey.parentPath, productKey.childName + "_Config"))
  from profileKey of getItem(joinPath(productKey.path, "Profile"))
  let version = getItemPropertyValue(profileKey, "BuildNum")
  let path = getItemPropertyValue(configKey, "ShellFolder")
  let name = "VisualStudio/" + version
  where testPath(joinPath(path, vsDevCmdPath))
  select new VisualStudioInstance(name, "Release", version, path, null)`

(the let group at the top just brings the functions into the scope of the query in the template. native syntax obviously wouldn't need that...)

If native, it would have just looked something like this:

let result =
    from productKey of getChildItem("HKCU...")
    from configKey of getItem(joinPath(productKey.parentPath, productKey.childName + "_Config"))
    from profileKey of getItem(joinPath(productKey.path, "Profile"))
    let version = getItemPropertyValue(profileKey, "BuildNum")
    let path = getItemPropertyValue(configKey, "ShellFolder")
    let name = "VisualStudio/" + version
    where testPath(joinPath(path, vsDevCmdPath))
    select new VisualStudioInstance(name, "Release", version, path, null);

each from (excluding the first), is essentially a flatMap. The transformation uses an object literal argument with each comprehension variable as a property. It uses iterable-query behind the scenes, so the above is translated to something like this:

import { Query } from "iterable-query";
let result = Query
  .from(getChildItem("HKCU..."))
  .selectMany(productKey => getItem(joinPath(productKey.parentPath, productKey.childName + "_Config")), (productKey, configKey) => ({ productKey, configKey }))
  .selectMany(({ productKey }) => getItem(joinPath(productKey.path, "Profile")), (env, profileKey) => ({ ...env, profileKey }))
  .select(({ profileKey, ...env }) => ({
      ...env,
      profileKey,
      version: getItemPropertyValue(profileKey, "BuildNum")
  }))
  .select(({ configKey, ...env }) => ({
      ...env,
      configKey,
      path: getItemPropertyValue(configKey, "ShellFolder")
  })
  .select(({ version, ...env }) => ({
      ...env,
      version,
      name: "VisualStudio/" + version
  })
  .where(({ path }) => testPath(joinPath(path, vsDevCmdPath)))
  .select(({ name, version, path }) => new VisualStudioInstance(name, "Release", version, path, null))

Something similar could be achieved with pipeline regardless of the topic variable, I think.

import { select, selectMany, where } from "@esfx/iter-fn";
let result = getChildItem("HKCU...")
    |> selectMany(%, productKey => getItem(joinPath(productKey.parentPath, productKey.childName + "_Config")), (productKey, configKey) => ({ productKey, configKey }))
    |> selectMany(%, ({ productKey }) => getItem(joinPath(productKey.path, "Profile")), (env, profileKey) => ({ ...env, profileKey }))
    |> select(%, ({ profileKey, ...env }) => ({
        ...env,
        profileKey,
        version: getItemPropertyValue(profileKey, "BuildNum"),
    }))
    |> select(%, ({ configKey, ...env }) => ({
        ...env,
        configKey,
        path: getItemPropertyValue(configKey, "ShellFolder")
    })
    |> select(%, ({ version, ...env }) => ({
        ...env,
        version,
        name: "VisualStudio/" + version
    })
    |> where(%, ({ path }) => testPath(joinPath(path, vsDevCmdPath)))
    |> select(%, ({ name, version, path }) => new VisualStudioInstance(name, "Release", version, path, null))

With context blocks resembling F# comprehension expressions:

let result = in @LINQ {
  const productKey =* getChildItem("HKCU…");
  const configKey =* getItem(joinPath(productKey.parentPath, productKey.childName + "_Config"));
  const profileKey =* getItem(joinPath(productKey.path, "Profile"));
  const rsion = getItemPropertyValue(getItemPropertyValue(profileKey, "BuildNum");
  const path = getItemPropertyValue(configKey, "ShellFolder");
  if (testPath(joinPath(path, vsDevCmdPath)))
    return new VisualStudioInstance(name, "Release", version, path, null);
};

I'm not sure I like the comprehension you wrote above for my example. It doesn't read as something that happens iteratively, so it feels like it could be easily confused or abused.

That's why I'm partial to the [FLWOR] (or, I guess "FLOWS") syntax of linq, since its obvious you're doing iteration, and easier to reason over how expensive an operation is.

The original generator comprehension syntax proposed for ES2015 felt somewhat unreadable and limited to me (i.e., using it for more than very simple comprehensions could make code difficult to understand).

LINQ does have its downsides though. If you want to do anything outside of from, let, where, orderby, join/join into, group/group into, or select/select into you're forced to parenthesize the expression, i.e.:

var x = (
  from num in numbers
  where num % 2 == 0
  select num
).Sum();

By the way, with regard to making it “obvious you’re doing integration”, computation expressions in F# do support customizing for loops too. So that example could be rewritten to use for. It would all be equivalent.

I'm not sure if that's better or worse, tbh. At that point I might as well have just written this:

const result = (function * () {
  for (const productKey of getChildItem("HKCU…"))
  for (const configKey of getItem(joinPath(productKey.parentPath, productKey.childName + "_Config")))
  for (const profileKey of getItem(joinPath(productKey.path, "Profile"))) {
    const version = getItemPropertyValue(getItemPropertyValue(profileKey, "BuildNum");
    const path = getItemPropertyValue(configKey, "ShellFolder");
    if (testPath(joinPath(path, vsDevCmdPath)))
      yield new VisualStudioInstance(name, "Release", version, path, null);
  }
})();

The upside of LINQ is brevity with a fair amount of expressivity, plus all of that "Expression tree" functionality that enabled XLinq, DLinq, etc.

Not to mention things like grouping and joins. The above is the most basic of comprehensions.

// C#
var result =
  from user in getUsers()
  join role in getRoles() on user.roleId equals role.id
  select new { user, role };

Ah, I guess they have custom operators. Damn that is going to be difficult to type check. Lets say you have this:

class QueryBuilder {
  @customOperation("existsNot")
  * existsNot<T>(iterable: Iterable<T>, predicate: (v: T) => boolean) {
    for (const x of iterable) if (!predicate(x)) yield x;
  }
}
const query = new QueryBuilder();
const result = in @query {
   for x of y
   existsNot(x) // have to look up the type of `query` and somehow associate the `customOperation` string as the name of an operator...
};

And aliasing customOperation to something else will make things harder. :(

I guess there are a lot more keywords than are listed above, based on the expression/translation table in https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/computation-expressions#creating-a-new-type-of-computation-expression

I don't know why but part of me wants it to be do@query { ... } instead of in @query { ... }, but that's not that important.

I could see that being a replacement for async do {}, i.e. in @Promise { ... } the biggest gotcha is that the expr in in @expr {} can completely change the meaning of the code, such that if the do block is long and you don't see the @expr you might not be able to clearly reason over what you're reading. (long as in, scrolled outside the editor viewport)

Yes. It’s not unlike handling await different in and out of async functions, but it’d be wider in scope as a potential problem. At least the ! in many of the keywords mitigate much of that.

re comprehensions - there's lots of references here: https://es.discourse.group/t/list-comprehension/112/3 i don't think that's something worth holding one's breath for.

I agree: Syntax that is specialized only for list comprehensions alone would indeed probably not be worth adding to the language.

In contrast, F#-style computation expressions would be more generally useful, with a single, unified syntax capturing many kinds of computations. These computations would include not only list comprehensions but also parser combinators, side-effect isolation, continuations and middleware, customized control flow and exception handling (“explaining” the existing generator/async-function syntax while enabling customizations like third-party future libraries and maybe even Mark Miller’s wavy-dot functionality), and many other DSLs.

I've never heard of computation expressions before. I'm trying to wrap my head around it.

It looks a bit like the idea I have of Algebraic Effects, but more specialized. Are they related in some way ?

They are indeed related, and I am exploring them at the same time too.

I recommend the following articles to learn about computation expressions:

• F# for Fun and Profit: Computation Expressions
• Syntax Matters: Writing abstract computations in F#
• F# Computation Expression Zoo, with the actual paper

(Warning: This Gist is in its very early days and it will probably be mid-2022 at the earliest before I’ll have this in a presentable shape. And a lot of old writing is hidden inside a “Scratchpad” disclosure element at the end.)