Top-Level Awaiting

For Godot

With your host

@MylesBorins

DotJS 2018

Bonjour!

My Name is Myles

I am gainfully employed by Google as a Developer Advocate

Focusing on the Node.js ecosystem and GCP

The opinions expressed in this talk are solely my own

Chapter One

Asynchronicity

is hard

Don't stop,

don't stop the thread

From the beginning of time (1995)

We had callbacks in JavaScript

a((aa) => {
    b(aa, (bb) => {
        c(bb, (cc) => {
            d(cc, (dd) => {
                e(dd, (OMG) => {
                    throw 'me into the abyss';
                });
            });
        });
    });
});

Promises

Made it into the language ~20 years later in 2014

a.then((aa) => {
  return b(aa);
}),then((bb) => {
  return c(bb);
}).then((cc) => {
  return d(cc);
}).then((OMG) => {
  throw 'me into the abyss';
}).catch((e) => {
  console.log('I got chu bae');
});

Async / Await

Arrived in December of 2016

(async function main () {
  try {
    const aa = await a();
    const bb = await b(aa);
    const cc = await c(bb);
    const OMG = await d(cc);
    throw 'me into the absyss';
  }
  catch (e) {
    console.log('never mind');
  }
}());

Chapter Two

Don't tell me

what I can't do

February 21 2017

Node.js 7.6.0 is release

It includes V8 5.5

which has support for async / await

I was really excited to use await until...

It looked like

Top-Level Await

Wasn't a thing

May 31 2017

My First attempt to make TLA work in node

a naive node patch to make TLA work that broke everything

Spoiler: It Didn't

It turned out that the Node.js test suite

Wasn't the only place that had an issue

With Top-Level Await

Chapter Three

Top-level await

is a footgun 👣🔫

In September of 2016

Rich Harris wrote

Top-Level Await is a footgun

https://gist.github.com/Rich-Harris/0b6f317657f5167663b493c722647221

a gist that would be brought up whenever i mentioned

Top-Level Await

TL;DR

Top-Level await can block execution
Fetching resources over network would blocked
No interop support for common.js
Cycles could introduce deadlock
Potential to introduce indeterminism into graph execution

Despite this strong critique

There was no prior decision from TC39 regarding Top-Level Await

I decided to take Top-Level Await to the committee

Worst case scenario, get a solid no and stop flame wars

Chapter Four

A history lesson

Top-Level await was always being considered

while async / await was being standardized

I did some research by reviewing the agendas of past meetings.

http://github.com/tc39/agendas/

Some quick history of

async / await

originally brought to TC39 in January of 2014
In April 2014 `await` keyword was reserved in the module goal
July 2015 `async` / `await` moved to stage two
it was decided to delay decision about Top-Level await at that time

taking a step back

The Spec is

ECMA-262

The Committee who implements it is

Consensus

The Stages

Stage 0: strawman

Stage 1: proposal

Stage 2: draft

Stage 3: candidate

Stage 4: finished

Chapter Five

A footgun in the door

In January 2018

Top-Level await was brought TC39

Attempting for Stage 1

The proposal included

History of TLA
Motivation for the feature
Potential Implementations
Use Cases
Constraints

Motivation #1

IIAFE

Immediately invoked async function expression

import static1 from './static1.mjs';
import { readFile } from 'fs';

(async () => {
  const dynamic = await import('./dynamic'
    + process.env.something + '.mjs');
  const file = JSON.parse(await readFile('./config.json'));
  // main program goes here...
})();

Motivation #2

Completely Async Modules

export default async () => {
  // import other modules like this one
  const import1 = await (await import('./import1.mjs')).default();
  const import2 = await (await import('./import2.mjs')).default();

  // compute some exports...
  return {
    export1: ...,
    export2: ...
  };
};

Potential Solutions

Variant A

A call to top-level await would block execution of the graph until it had resolved.

Assuming A, B, C

all have a Top-Level await

import a from './a.mjs'
import b from './b.mjs'
import c from './c.mjs'

console.log(a, b, c);

The code is equivalent to

(async () => {
  const a = await import('./a.mjs');
  const b = await import('./b.mjs');
  const c = await import('./c.mjs');

  console.log(a, b, c);
})();

Variant B

A call to top-level await would block execution of child nodes in the graph but would allow siblings to continue to execute.

Assuming A, B, C

all have a Top-Level await

import a from './a.mjs'
import b from './b.mjs'
import c from './c.mjs'

console.log(a, b, c);

The code is equivalent to

(async () => {
  const [a, b, c] = await Promise.all([
    import('./a.mjs'),
    import('./b.mjs'),
    import('./c.mjs')
  ]);
  console.log(a, b, c);
})();

Optional Constraint

Enforcing that top-level await could only be used inside of a module without exports.

Use Cases

Dynamic dependency pathing

export const strings =
  await import(`/i18n/${navigator.language}`);

Resource initialization

const connection = await dbConnector();

Dependency fallbacks

let jQuery;
try {
  jQuery = await import('https://cdn-a.com/jQuery');
} catch {
  jQuery = await import('https://cdn-b.com/jQuery');
}

Constraints

Variant A would halt progress in the module graph until resolved
Variant B would halt progress but not block siblings execution
The optional constraint would limit the above concerns

There are existing ways to halt progress in JavaScript

Infinite Loops

for (const n of primes()) {
  console.log(`${n} is prime}`);
}

Infinite Recursion

const fibb = n => (n ? fibb(n - 1) : 1);
fibb(Infinity);

Atomics.wait

Atomics.wait(shared_array_buffer, 0, 0);

Atomics allow blocking forward progress by waiting on an index that never changes

Is Blocking Execution

A strong enough reason

To block Top-Level Await?

Another Constraint:

Deadlock

Achievement unlocked

Top-Level Await

was moved to Stage 1

Chapter Six

The Road

To Stage 2

Reaching stage 1 was a big step

It signaled that TC39 was interested

In exploring the problem space of TLA

Moving to stage 2 requires reaching consensus

On the general shape of the solution to the problem

We identified necessary

spec changes

When your ESM code is executed

It is represented

as a graph of

Source Text Module Records

A Source Text Module Record is used to represent information about a module that was defined from ECMAScript source text that was parsed using the goal symbol Module.

ECMA262 defines how these Records are instantiated

and how they are represented in memory

We made all modules in the graph

Async

To do this we abstracted code from the

Async Function Specification

for instantiation of Module Records in the graph

With this infrastructure in place

Every module would now have a promise capability

Allowing for deferred execution at the Top-Level

We were able to defer execution by calling

Spec-Level Await on the execution of the module

Yes, the spec has it's own abstractions

this algorithm supports Variant A

A spec level Promise.all would add support

for Variant B

as such we were able to defer

making a decision

on A versus B

Unblocking Objections to either semantics

We clarified that the feature would only be supported

In the module goal

Unblocking objections about interoperability

We deferred to current behavior

For handling deadlock

Unblocking objections related to cycles

In may of 2018

Top-Level Await

went to stage 2

Chapter Seven

What's next?

I want to move forward

With Variant B

We need to build consensus around these semantics

We are going through a variety of examples

To define semantics for deadlock

The intent is to make things fail early and obviously

We need to finish writing the spec text

The text then needs to be reviewed and approved

ETA, ¯\_(ツ)_/¯

const [you, me] = Promise.all([
  await topLevelAwait('patience')
  await topLevelAwait('persistence');
]);

Thank You

@MylesBorins