No promises: asynchronous JavaScript with only generators

[2015-03-04] esnext, dev, javascript

Two ECMAScript 6 ^[1] features enable an intriguing new style of asynchronous JavaScript code: promises ^[2] and generators ^[3]. This blog post explains this new style and presents a way of using it without promises.

Overview

Normally, you make a function calls like this:

let result = func(···);
console.log(result);

It would be great if this style of invocation also worked for functions that perform tasks (such as downloading a file) asynchronously. For that to work, execution of the previous code would have to pause until func() returns with a result.

Before ECMAScript 6, you couldn’t pause and resume the execution of code, but you could simulate it, by putting console.log(result) into a callback, a so-called continuation ^[4]. The continuation is triggered by asyncFunc(), once it is done:

asyncFunc('http://example.com', result => {
    console.log(result);
});

Promises ^[2:1] are basically a smarter way of managing callbacks:

asyncFunc('http://example.com')
.then(result => {
    console.log(result);
});

In ECMAScript 6, you can use generator functions ^[3:1], which can be paused and resumed. With a library such as Q, a generator-based solution looks almost like our ideal code:

Q.spawn(function* () {
    let result = yield asyncFunc('http://example.com');
    console.log(result);
});

Alas, asyncFunc() needs to be implemented using promises:

function asyncFunc(url) {
    return new Promise((resolve, reject) => {
        otherAsyncFunc(url,
            result => resolve(result));
    });
}

However, with a small library shown later, you can run the initial code like with Q.spawn(), but implement asyncFunc() like this:

function* asyncFunc(url) {
    const caller = yield; // (A)
    otherAsyncFunc(url,
        result => caller.success(result));
}

Line A is how the library provides asyncFunc() with callbacks. The advantage compared to the previous code is that this function is again a generator and can make other asynchronous calls via yield.

Code

I’ll first show two examples, before I present the code of the library.

Example 1: `echo()`

echo() is an asynchronous function, implemented via a generator:

function* echo(text, delay = 0) {
    const caller = yield;
    setTimeout(() => caller.success(text), delay);
}

In the following code, echo() is used three time, sequentially:

run(function* echoes() {
    console.log(yield echo('this'));
    console.log(yield echo('is'));
    console.log(yield echo('a test'));
});

The parallel version of this code looks as follows.

run(function* parallelEchoes() {
    let startTime = Date.now();
    let texts = yield [
        echo('this', 1000),
        echo('is', 900),
        echo('a test', 800)
    ];
    console.log(texts); // ['this', 'is', 'a test']
    console.log('Time: '+(Date.now()-startTime));
});

As you can see, the library performs the asynchronous calls in parallel if you yield an array of generator invocations.

This code takes about 1000 milliseconds.

Example 2: `httpGet()`

The following code demonstrates how you can implement a function that gets a file via XMLHttpRequest:

function* httpGet(url) {
    const caller = yield;

    var request = new XMLHttpRequest();
    request.onreadystatechange = function () {
        if (this.status === 200) {
            caller.success(this.response);
        } else {
            // Something went wrong (404 etc.)
            caller.failure(new Error(this.statusText));
        }
    }
    request.onerror = function () {
        caller.failure(new Error(
            'XMLHttpRequest Error: '+this.statusText));
    };
    request.open('GET', url);
    request.send();    
}

Let’s use httpGet() sequentially:

run(function* downloads() {
    let text1 = yield httpGet('https://localhost:8000/file1.html');
    let text2 = yield httpGet('https://localhost:8000/file2.html');
    console.log(text1, text2);
});

Using httpGet() in parallel looks like this:

run(function* parallelDownloads() {
    let [text1,text2] = yield [
        httpGet('https://localhost:8000/file1.html'),
        httpGet('https://localhost:8000/file2.html')
    ];
    console.log(text1, text2);
});

The library

The library profits from the fact that calling a generator function does not execute its body, but returns a generator object.

/**
 * Run the generator object `genObj`,
 * report results via the callbacks in `callbacks`.
 */
function runGenObj(genObj, callbacks = undefined) {
    handleOneNext();

    /**
     * Handle one invocation of `next()`:
     * If there was a `prevResult`, it becomes the parameter.
     * What `next()` returns is what we have to run next.
     * The `success` callback triggers another round,
     * with the result assigned to `prevResult`.
     */
    function handleOneNext(prevResult = null) {
        try {
            let yielded = genObj.next(prevResult); // may throw
            if (yielded.done) {
                if (yielded.value !== undefined) {
                    // Something was explicitly returned:
                    // Report the value as a result to the caller
                    callbacks.success(yielded.value);
                }
            } else {
                setTimeout(runYieldedValue, 0, yielded.value);
            }
        }
        // Catch unforeseen errors in genObj
        catch (error) {
            if (callbacks) {
                callbacks.failure(error);
            } else {
                throw error;
            }
        }
    }
    function runYieldedValue(yieldedValue) {
        if (yieldedValue === undefined) {
            // If code yields `undefined`, it wants callbacks
            handleOneNext(callbacks);
        } else if (Array.isArray(yieldedValue)) {
            runInParallel(yieldedValue);
        } else {
            // Yielded value is a generator object
            runGenObj(yieldedValue, {
                success(result) {
                    handleOneNext(result);
                },
                failure(err) {
                    genObj.throw(err);
                },
            });
        }
    }

    function runInParallel(genObjs) {
        let resultArray = new Array(genObjs.length);
        let resultCountdown = genObjs.length;
        for (let [i,genObj] of genObjs.entries()) {
            runGenObj(genObj, {
                success(result) {
                    resultArray[i] = result;
                    resultCountdown--;
                    if (resultCountdown <= 0) {
                        handleOneNext(resultArray);
                    }
                },
                failure(err) {
                    genObj.throw(err);
                },
            });
        }
    }
}

function run(genFunc) {
    runGenObj(genFunc());
}

Note that you only need use caller = yield and caller.success(···) in asynchronous functions that use callbacks. If an asynchronous function only calls other asynchronous functions (via yield) then you can simply explicitly return a value.

One important feature is missing: support for calling async functions implemented via promises. It would be easy to add, though – by adding another case to runYieldedValue().

Conclusion: asynchronous JavaScript via coroutines

Couroutines ^[5] are a single-threaded version of multi-tasking: Each coroutine is a thread, but all coroutines run in a single thread and they explicitly relinquish control via yield. Due to the explicit yielding, this kind of multi-tasking is also called cooperative (versus the usual preemptive multi-tasking).

Generators are shallow co-routines ^[6]: their execution state is only preserved within the generator function: It doesn’t extend further backwards than that and recursively called functions can’t yield.

The code for asynchronous JavaScript without promises that you have seen in this blog post is purely a proof of concept. It is completely unoptimized and may have other flaws preventing it from being used in practice.

But coroutines seem like the right mental model when thinking about asynchronous computation in JavaScript. They could be an interesting avenue to explore for ECMAScript 2016 (ES7) or later. As we have seen, not much would need to be added to generators to make this work:

caller = yield is a kludge.
Similarly, having to report results and errors via callbacks is unfortunate. It’d be nice if return and throw could always be used, but they don’t work inside callbacks.

What about streams?

When it comes to asynchronous computation, there are two fundamentally different needs:

The results of a single computation: One popular way of performing those are promises.
A series of results: Asynchronous Generators ^[7] have been proposed for ECMAScript 2016 for this use case.

For #1, coroutines are an interesting alternative. For #2, David Nolen has suggested ^[8] that CSP (Communicating Sequential Processes) work well. For binary data, WHATWG is working on Streams ^[9].

Current practical solutions

All current practical solutions are based on Promises:

Q is a promise library and polyfill that includes the aforementioned Q.spawn(), which is based on promises.
co brings just the spawn() functionality and relies on an external Promise implementation. It is therefore a good fit for environments such as Babel that already have Promises.
Babel has a first implementation of async functions (as proposed for ECMAScript 2016). Under the hood, they are translated to code that is similar to spawn() and based on Promises. However, if you use this feature, you are leaving standard territory and your code won’t be portable to other ES6 environments. Async functions may still change considerably before they are standardized.

No promises: asynchronous JavaScript with only generators

Overview #

Code #

Example 1: echo() #

Example 2: httpGet() #

The library #

Conclusion: asynchronous JavaScript via coroutines #

What about streams? #

Current practical solutions #

Further reading #