With the introduction of ES6, we acquired the support for synchronously iterating over data. We could, of course, already iterate over iterable built-in structures like objects or arrays, but the big introduction was the formalization of an implementable interface to create both our iterables and generators.

But what about the scenarios where our iterations are done over data that is obtained from an asynchronous source, such as a set of remote HTTP calls or reading from a file?

In this article, we will conduct a practical analysis of the “Asynchronous Iteration” proposal, which is intended to add:

“support for asynchronous iteration using the AsyncIterable and AsyncIterator protocols. It introduces a new IterationStatement, for-await-of, and adds syntax for creating async generator functions and methods.”

For this guide, only basic knowledge of JavaScript (or programming in general) is required. All our examples, which will be presented with some simple TypeScript annotations, can be found in this GitHub repository.

Recap of Synchronous Iterators and Generators

In this section, we will do a quick review of synchronous iterators and generators in JavaScript so we can more easily extrapolate them into the asynchronous case.

What is an iteration anyway?

Let us look into this question by self-realizing what we usually have at hand when iterating. For example, on one side, we have our arrays, strings, etc. being basically our sources. On the other side, we have what we usually use as our means to consume our data via an iteration — namely our for loops, spread operators, etc.

Based on this, we can look at an iteration as a protocol that, when implemented by our sources, will allow consumers to sequentially “consume” its contents using a set of regular operations. This protocol could then be represented by the following interface:

So, putting it verbosely for those readers who may not be familiar with TS interface descriptions:

• A SynchronousIterable provides a method via a Symbol.iterator that would return a SynchronousIterator.
• Our SynchronousIterator would then return our IteratorResults from its implementation of the .next() method.
• The IteratorResults would then contain a value to hold the current iterated value as well as a done flag that is set to true after the last item is iterated through (and false while iterating).

Note: You can find out more about this by reading the ECMAScript 2021 Language Specification documentation.

An example of using this interface can be easily showcased by manually iterating over an array:

Sources that implement this interface can also be iterated via a for..of iteration directive that you have probably made use of at some point:

Of course, we would expect sources like the built-in array (as used above) to be iterable naturally. To then showcase it differently, we’ll implement that interface, for example, to generate a range of numbers:

What about generators?

Usually, functions return either a single value or none. We can think of generators as entities that can return, in sequence, multiple values. To this holding of values, it was attributed the concept of yielding.

These generator functions do not behave as regular functions, as they are lazily evaluated. So when called, they will return you a generator object that will be responsible for managing its execution. These generators are also iterable, meaning they also implement our interface so we can actually loop over them similarly as above:

Also, it is also very important to denote that the .next() function is key to obtaining the next yielded value for these generator objects. This will then produce the expected outputs.

As we can now yield values (instead of state), we can then re-implement our iterable range from Figure 1.4, but now using a generator function:

As we can see, we can now leverage a generator function to simplify our original ranged iterable.

An Asynchronous Interface Proposal

After grasping the idea behind the interface definition of an iterable (Figure 1.1), it would be easy to now extend it in a way where every step of our iteration would then return the result of an asynchronous operation. The usual representation of this is via promises:

From the definition above, we can easily identify that the asynchronous operation is indeed when providing the .next() element of the iteration. Therefore, it is trivial to proceed with implementing it in a way that handles the results as promises. Let us make this clear by adapting our ranged iterator (from Figure 1.5) in this way with a faux delay:

We can use this concept to actually abstract our generator in (1.7). It would then be trivial to implement the same range using an asynchronous generator:

Regardless of how we generate the data, we may simply iterate over the elements as if an asynchronous source was yet another iterable. Actually, it is as long as it implements our interface.

To demonstrate this, the next section will use an asynchronous source (a Hacker News top stories feed) that we will then use to manipulate as we would with any other iterable structure.

Practical Exercise: a HackerNews Iterable

Based on the implementation of an asynchronous generator (2.3), it is now trivial to materialize a generator source for our posts. For this example, we will use the HN API:

This code simply tucks the asynchronous logic by implementing still the usual interface for an async iterator. For this example, we are limiting the entries to iterate over, which is optional. By yielding each iterated value, we can easily consume this source with the following simple implementation:

As expected, this loops and renders a list of comments for our source:

We can now look at our data source and handle it as a simple data sequence, keeping the full asynchronous fetching and manipulation logic within our generator definition.

Summary

Hopefully, this article showcases that it is trivial to transform and observe our asynchronous data sources as well as iterables by applying simple language formalities already available in the language specification.

By generalizing the synchronous case to also cover asynchronous generation, we can now iterate over any iterable source regardless of the nature of the data source — as long as it implements our interface. Looking at our asynchronous data sources as iterables opens a creative potential for our ideas towards more idiomatic and eloquent codebases.

For all the code examples, please refer to this GitHub repo.