Oscar Funes' Blog

Generators and Iterators

July 15, 2019

The topic of Generators (and iterators) has always made me curious. Every time I searched on the subject, it was a Fibonacci example.

function* fib (n) {
  const isInfinite = n === undefined;
  let current = 0;
  let next = 1;

  while (isInfinite || n--) {
    yield current;
    [current, next] = [next, current + next];
  }
}

What happens here is a function that you can call an “infinite” number for times to get the next number of the Fibonacci sequence.

However, the Generator function, denoted by the *, is a constructor for an Iterator. What is an Iterator?

Iterator

An Iterator is an object with a next() method. The return value of the method should be an object with two properties: value and done.

The value property is the next value in the series. The done property is a flag that tells the consumer if the Iterator has completed or not. A value can be returned when the done flag is set to true.

The above description is what is known as the Iterable Protocol. There is some feature of the language that needs the iterable protocol to work correctly.

The most used could be the for ... of loop and the spread syntax. Certain constructors for collections like Map, WeakMap, Set, WeakSet, Array.from(). Also, Promise methods like Promise.all or Promise.race.

Think of this as: When you would want an object to behave like an array, you can use the iterator protocol to make them fit.

Example of usage

Going back to the Fibonacci example, we could destructure the first N values from the series like this:

const [first, second, third] = fib();
console.log(first, second, third) // 0 1 1

const [...sequence] = fib(); // This will explode!

The second example will explode due to the infinite nature of the Fibonacci generator we wrote. It will try to fill the sequence by pulling values from the iterator until it is done: true, which won’t happen.

Different way to access the values is through a for ... of loop. Like this:

for (const n of fib()) {
  console.log(n); // this will log until Infinity
}

While the previous examples are interesting, they don’t seem useful in day to day work. How do I make generators work for me?

Enter the @@iterator Symbol.

@@iterator

The @@iterator accessible through Symbol.iterator is a well-known symbol. This symbol can be used to define a property in a custom object to define iterator behavior.

This means that by adding the property to an object, we can make it behave like an array.

For example:

const firstThreeNumbers = { one: 1, two: 2, three: 3 };
// defining an iterator
firstThreeNumbers[Symbol.iterator] = function * () {
    yield this.one;
    yield this.two;
    yield this.three;
};

console.log(...firstThreeNumbers); // 1, 2, 3
console.log(await Promise.all(firstThreeNumbers)); // [1, 2, 3]

This makes the usage of this more interesting, especially if you’re building custom collection types in your source code and want them to be treated as arrays by native language features.

There’s also another way to iterate the values of the iterator, and that is manually using the iterator object.

Manual access

The “manual” access means, creating the iterator first and then calling iterator.next() for the value.

const sequence = fib();
console.log(sequece.next().value); // 0
console.log(sequece.next().value); // 1
console.log(sequece.next().value); // 1
console.log(sequece.next().value); // 2

This doesn’t seem that useful? The only advantage of accessing the values through this method is that the next() method accepts a value. So if you do another generator to consider this, it could look like this:

function* random() {
    let forever = true;
    let value = 0;
    while (forever) {
    const stop = yield value++;
    forever = !stop;
    }
}

We can use the declared generator as in the first example:

const sequence = random();
const sequence = random();
console.log(sequence.next().value); // 0
console.log(sequence.next().value); // 1
console.log(sequence.next(true).value); // undefined
console.log(sequence.next().value); // undefined

Delegation

The generators support delegating to an inner generator through the usage of yield*. As I mentioned, when creating a class that you want to treat as an array, and internally has an array, something like:

class Pages {
  constructor(...pages) {
    this._pages = pages;
  }
  *[Symbol.iterator]() {
    yield* this._pages;
  }
}

The yield* expression will delegate to the inner array iterator the yielding of values.

const pages = new Pages(1,2,3,4,5);
const sequence = pages[Symbol.iterator]();
console.log(sequence.next().value);
console.log(sequence.next().value);
console.log(sequence.next().value);
console.log(sequence.next().value);
console.log(sequence.next().value);

// OR with for ... of loop
const pages = new Pages(1,2,3,4,5);
for (const page of pages) {
console.log(page);
}

Conclusion

While theoretically interesting, I still haven’t found a use for the iterator besides treating custom classes/objects as arrays through native functions of the language.

Any time you want to iterate through a list, in a serial way, you could use this function of the language.

You could also yield Promise and wait for it to resolve and pass the value again through the next() parameter.

Using promises here would allow going through an iterator of promises one by one in a for ... of loop. Imagine querying a paged API, or querying in a Database using Knex or similar libraries.

Recently Javascript added Symbol.asyncIterator which allows writing iterators using async/await functionality.

If you’ve found ways to use the feature, please let me know!

Happy coding!


Oscar Funes

I love distributed systems, software architecture, emotional intelligence and how they intersect.