Iterator and the for...of Loop

The Concept of Iterator (Traverser)

The original data structures in JavaScript for representing "collections" were mainly arrays (Array) and objects (Object). ES6 added Map and Set. With four types of data collections now available, and users able to combine them and define their own data structures (for example, arrays whose members are Maps, or Maps whose members are objects), a unified interface mechanism is needed to handle all the different data structures. An Iterator is precisely such a mechanism. It is an interface that provides a unified access mechanism for different data structures. Any data structure that deploys the Iterator interface can complete traversal operations (i.e., processing all members of the data structure sequentially).

Iterator serves three purposes: first, to provide a unified, convenient access interface for various data structures; second, to enable the members of a data structure to be arranged in some order; and third, ES6 created a new traversal command, the for...of loop, and the Iterator interface is mainly consumed by for...of.

The traversal process of an Iterator works as follows.

(1) Create a pointer object pointing to the start of the current data structure. In other words, a traverser object is essentially a pointer object.

(2) The first call to the pointer object's next method moves the pointer to the first member of the data structure.

(3) The second call to the pointer object's next method moves the pointer to the second member of the data structure.

(4) Continue calling the pointer object's next method until it points to the end of the data structure.

Each call to next returns information about the current member of the data structure. Specifically, it returns an object containing two properties: value and done. The value property is the value of the current member, and the done property is a boolean indicating whether traversal has ended.

Here is an example simulating the return value of the next method.

var it = makeIterator(['a', 'b']);

it.next() // { value: "a", done: false }
it.next() // { value: "b", done: false }
it.next() // { value: undefined, done: true }

function makeIterator(array) {
  var nextIndex = 0;
  return {
    next: function() {
      return nextIndex < array.length ?
        {value: array[nextIndex++], done: false} :
        {value: undefined, done: true};
    }
  };
}

The code above defines a makeIterator function, which is a traverser generator function whose purpose is to return a traverser object. Executing this function on the array ['a', 'b'] returns the array's traverser object (i.e., pointer object) it.

The pointer object's next method moves the pointer. Initially, the pointer points to the beginning of the array. Then, each call to next moves the pointer to the next member of the array. The first call points to a; the second call points to b.

The next method returns an object representing the current data member's information. This object has two properties: value returns the current member, and done is a boolean indicating whether traversal has ended -- that is, whether it's necessary to call next again.

In short, calling the pointer object's next method traverses the given data structure.

For the traverser object, the done: false and value: undefined properties can both be omitted, so the makeIterator function above can be simplified as follows.

function makeIterator(array) {
  var nextIndex = 0;
  return {
    next: function() {
      return nextIndex < array.length ?
        {value: array[nextIndex++]} :
        {done: true};
    }
  };
}

Since Iterator only adds the interface specification to the data structure, the traverser and the data structure it traverses are actually separate. It's entirely possible to create a traverser object without a corresponding data structure, or to simulate a data structure with a traverser object. Here is an example of an infinitely running traverser object.

var it = idMaker();

it.next().value // 0
it.next().value // 1
it.next().value // 2
// ...

function idMaker() {
  var index = 0;

  return {
    next: function() {
      return {value: index++, done: false};
    }
  };
}

In the example above, the traverser generator function idMaker returns a traverser object (i.e., pointer object) that has no corresponding data structure -- or rather, the traverser object itself describes a data structure.

Using TypeScript notation, the specifications for the iterator interface (Iterable), pointer object (Iterator), and the return value of the next method can be described as follows.

interface Iterable {
  [Symbol.iterator]() : Iterator,
}

interface Iterator {
  next(value?: any) : IterationResult,
}

interface IterationResult {
  value: any,
  done: boolean,
}

The Default Iterator Interface

The purpose of the Iterator interface is to provide a unified access mechanism for all data structures, namely the for...of loop (see below). When a for...of loop is used to traverse a data structure, the loop automatically looks for the Iterator interface.

A data structure is considered "iterable" as long as it has deployed the Iterator interface (iterable).

ES6 specifies that the default Iterator interface is deployed on a data structure's Symbol.iterator property. In other words, a data structure can be considered "iterable" as long as it has a Symbol.iterator property. The Symbol.iterator property itself is a function -- the default traverser generator function for the current data structure. Executing this function returns a traverser. As for the property name Symbol.iterator, it is an expression that returns the iterator property of the Symbol object. This is a predefined, Symbol-type special value, which is why it must be placed in brackets (see the "Symbol" chapter).

const obj = {
  [Symbol.iterator] : function () {
    return {
      next: function () {
        return {
          value: 1,
          done: true
        };
      }
    };
  }
};

In the code above, object obj is iterable because it has a Symbol.iterator property. Executing this property returns a traverser object. The fundamental characteristic of this object is that it has a next method. Each call to next returns an information object representing the current member, with value and done properties.

Some ES6 data structures natively have the Iterator interface (such as arrays) -- they can be traversed by for...of without any processing. This is because these data structures natively deploy the Symbol.iterator property (see below), while other data structures (such as objects) do not. Any data structure that has deployed the Symbol.iterator property is said to have deployed the iterator interface. Calling this interface returns a traverser object.

Data structures that natively have the Iterator interface are as follows.

• Array
• Map
• Set
• String
• TypedArray
• The arguments object of functions
• NodeList objects

Here is an example of the Symbol.iterator property of an array.

let arr = ['a', 'b', 'c'];
let iter = arr[Symbol.iterator]();

iter.next() // { value: 'a', done: false }
iter.next() // { value: 'b', done: false }
iter.next() // { value: 'c', done: false }
iter.next() // { value: undefined, done: true }

In the code above, variable arr is an array that natively has the iterator interface, deployed on arr's Symbol.iterator property. So calling this property gives us the traverser object.

For data structures that natively deploy the Iterator interface, you don't need to write your own traverser generator function -- for...of will automatically traverse them. For other data structures (mainly objects), the Iterator interface needs to be deployed on the Symbol.iterator property for them to be traversed by for...of.

The reason Objects don't have a default Iterator interface is that the order of property traversal is uncertain and needs to be manually specified by the developer. Essentially, a traverser is a form of linear processing, and for any non-linear data structure, deploying an iterator interface is equivalent to deploying a linear transformation. Strictly speaking, deploying an iterator interface on objects is not very necessary, because at that point objects are effectively being used as Map structures, and ES5 didn't have Map structures while ES6 natively provides them.

For an object to be callable by for...of, it must deploy a traverser generator method on the Symbol.iterator property (or have it on a prototype chain object).

class RangeIterator {
  constructor(start, stop) {
    this.value = start;
    this.stop = stop;
  }

  [Symbol.iterator]() { return this; }

  next() {
    var value = this.value;
    if (value < this.stop) {
      this.value++;
      return {done: false, value: value};
    }
    return {done: true, value: undefined};
  }
}

function range(start, stop) {
  return new RangeIterator(start, stop);
}

for (var value of range(0, 3)) {
  console.log(value); // 0, 1, 2
}

The code above is a class deploying the Iterator interface. The Symbol.iterator property corresponds to a function that returns the current object's traverser object when executed.

Here is an example implementing a pointer structure using a traverser.

function Obj(value) {
  this.value = value;
  this.next = null;
}

Obj.prototype[Symbol.iterator] = function() {
  var iterator = { next: next };

  var current = this;

  function next() {
    if (current) {
      var value = current.value;
      current = current.next;
      return { done: false, value: value };
    } else {
      return { done: true };
    }
  }
  return iterator;
}

var one = new Obj(1);
var two = new Obj(2);
var three = new Obj(3);

one.next = two;
two.next = three;

for (var i of one){
  console.log(i); // 1, 2, 3
}

The code above first deploys the Symbol.iterator method on the constructor's prototype chain. Calling this method returns the traverser object iterator. Calling the object's next method returns a value while automatically moving the internal pointer to the next instance.

Here is another example of adding an Iterator interface to an object.

let obj = {
  data: [ 'hello', 'world' ],
  [Symbol.iterator]() {
    const self = this;
    let index = 0;
    return {
      next() {
        if (index < self.data.length) {
          return {
            value: self.data[index++],
            done: false
          };
        } else {
          return { value: undefined, done: true };
        }
      }
    };
  }
};

For array-like objects (those with numeric key names and a length property), there is a simple way to deploy the Iterator interface: have the Symbol.iterator method directly reference the array's Iterator interface.

NodeList.prototype[Symbol.iterator] = Array.prototype[Symbol.iterator];
// Or
NodeList.prototype[Symbol.iterator] = [][Symbol.iterator];

[...document.querySelectorAll('div')] // Now works

NodeList objects are array-like objects that already have a traversal interface and can be directly traversed. In the code above, we change their traversal interface to the array's Symbol.iterator property, and you can see it has no effect.

Here is another example of an array-like object using the array's Symbol.iterator method.

let iterable = {
  0: 'a',
  1: 'b',
  2: 'c',
  length: 3,
  [Symbol.iterator]: Array.prototype[Symbol.iterator]
};
for (let item of iterable) {
  console.log(item); // 'a', 'b', 'c'
}

Note that deploying the array's Symbol.iterator method on a regular object has no effect.

let iterable = {
  a: 'a',
  b: 'b',
  c: 'c',
  length: 3,
  [Symbol.iterator]: Array.prototype[Symbol.iterator]
};
for (let item of iterable) {
  console.log(item); // undefined, undefined, undefined
}

If the Symbol.iterator method does not correspond to a traverser generator function (i.e., it doesn't return a traverser object), the interpreter will throw an error.

var obj = {};

obj[Symbol.iterator] = () => 1;

[...obj] // TypeError: [] is not a function

In the code above, variable obj's Symbol.iterator method does not correspond to a traverser generator function, so it throws an error.

Once the traverser interface is in place, data structures can be traversed with for...of (see below), as well as with while loops.

var $iterator = ITERABLE[Symbol.iterator]();
var $result = $iterator.next();
while (!$result.done) {
  var x = $result.value;
  // ...
  $result = $iterator.next();
}

In the code above, ITERABLE represents some iterable data structure, and $iterator is its traverser object. Each time the traverser object moves its pointer (next method), it checks the done property of the return value. If traversal hasn't ended, it moves the traverser object's pointer to the next step (next method), repeating the cycle.

Situations That Call the Iterator Interface

Some situations automatically call the Iterator interface (i.e., the Symbol.iterator method). Besides the for...of loop introduced below, there are several other situations.

(1) Destructuring Assignment

When destructuring arrays and Set structures, the Symbol.iterator method is called by default.

let set = new Set().add('a').add('b').add('c');

let [x,y] = set;
// x='a'; y='b'

let [first, ...rest] = set;
// first='a'; rest=['b','c'];

(2) Spread Operator

The spread operator (...) also calls the default Iterator interface.

// Example 1
var str = 'hello';
[...str] //  ['h','e','l','l','o']

// Example 2
let arr = ['b', 'c'];
['a', ...arr, 'd']
// ['a', 'b', 'c', 'd']

The spread operator in the code above internally calls the Iterator interface.

In practice, this provides a convenient mechanism to convert any data structure that has deployed the Iterator interface into an array. In other words, as long as a data structure has deployed the Iterator interface, the spread operator can be used to convert it to an array.

let arr = [...iterable];

(3) yield*

yield* is followed by an iterable structure, and it calls that structure's traverser interface.

let generator = function* () {
  yield 1;
  yield* [2,3,4];
  yield 5;
};

var iterator = generator();

iterator.next() // { value: 1, done: false }
iterator.next() // { value: 2, done: false }
iterator.next() // { value: 3, done: false }
iterator.next() // { value: 4, done: false }
iterator.next() // { value: 5, done: false }
iterator.next() // { value: undefined, done: true }

(4) Other Situations

Since array traversal calls the traverser interface, any situation that accepts an array as a parameter actually calls the traverser interface. Here are some examples:

• for...of
• Array.from()
• Map(), Set(), WeakMap(), WeakSet() (e.g., new Map([['a',1],['b',2]]))
• Promise.all()
• Promise.race()

The String Iterator Interface

Strings are array-like objects that natively have the Iterator interface.

var someString = "hi";
typeof someString[Symbol.iterator]
// "function"

var iterator = someString[Symbol.iterator]();

iterator.next()  // { value: "h", done: false }
iterator.next()  // { value: "i", done: false }
iterator.next()  // { value: undefined, done: true }

In the code above, calling the Symbol.iterator method returns a traverser object. On this traverser, you can call the next method to traverse the string.

You can override the native Symbol.iterator method to modify traverser behavior.

var str = new String("hi");

[...str] // ["h", "i"]

str[Symbol.iterator] = function() {
  return {
    next: function() {
      if (this._first) {
        this._first = false;
        return { value: "bye", done: false };
      } else {
        return { done: true };
      }
    },
    _first: true
  };
};

[...str] // ["bye"]
str // "hi"

In the code above, the string str's Symbol.iterator method has been modified, so the spread operator (...) returns bye instead, while the string itself is still hi.

The Iterator Interface and Generator Functions

The simplest implementation of the Symbol.iterator method is to use Generator functions, which will be introduced in the next chapter.

let myIterable = {
  [Symbol.iterator]: function* () {
    yield 1;
    yield 2;
    yield 3;
  }
}
[...myIterable] // [1, 2, 3]

// Or using the shorthand syntax

let obj = {
  * [Symbol.iterator]() {
    yield 'hello';
    yield 'world';
  }
};

for (let x of obj) {
  console.log(x);
}
// "hello"
// "world"

In the code above, the Symbol.iterator method requires almost no code deployment -- just use the yield command to provide each step's return value.

The Traverser Object's return() and throw() Methods

In addition to the next method, a traverser object can also have return and throw methods. If you write your own traverser object generator function, next is required, while return and throw are optional.

The return method is used when a for...of loop exits early (usually due to an error or a break statement). If an object needs cleanup or resource release before traversal completes, you can deploy the return method.

function readLinesSync(file) {
  return {
    [Symbol.iterator]() {
      return {
        next() {
          return { done: false };
        },
        return() {
          file.close();
          return { done: true };
        }
      };
    },
  };
}

In the code above, the function readLinesSync accepts a file object as a parameter and returns a traverser object that deploys both a next method and a return method. The following two situations will both trigger the return method.

// Situation 1
for (let line of readLinesSync(fileName)) {
  console.log(line);
  break;
}

// Situation 2
for (let line of readLinesSync(fileName)) {
  console.log(line);
  throw new Error();
}

In the code above, Situation 1 outputs the first line of the file and then executes the return method to close the file. Situation 2 executes the return method to close the file and then throws the error.

Note that return must return an object, as required by the Generator specification.

The throw method is mainly used with Generator functions and is not typically needed for regular traverser objects. Please refer to the "Generator Functions" chapter.

The for...of Loop

ES6 borrows from C++, Java, C#, and Python by introducing the for...of loop as a unified method for traversing all data structures.

A data structure is considered to have an iterator interface if it has deployed the Symbol.iterator property, and it can be traversed with for...of. In other words, the for...of loop internally calls the data structure's Symbol.iterator method.

The for...of loop can be used with arrays, Set and Map structures, certain array-like objects (such as arguments objects, DOM NodeList objects), Generator objects (discussed later), and strings.

Arrays

Arrays natively have the iterator interface (i.e., the Symbol.iterator property is deployed by default). A for...of loop essentially calls the traverser produced by this interface. This can be proven with the following code.

const arr = ['red', 'green', 'blue'];

for(let v of arr) {
  console.log(v); // red green blue
}

const obj = {};
obj[Symbol.iterator] = arr[Symbol.iterator].bind(arr);

for(let v of obj) {
  console.log(v); // red green blue
}

In the code above, the empty object obj deploys the Symbol.iterator property of array arr. The result is that obj's for...of loop produces exactly the same results as arr.

The for...of loop can replace the array instance's forEach method.

const arr = ['red', 'green', 'blue'];

arr.forEach(function (element, index) {
  console.log(element); // red green blue
  console.log(index);   // 0 1 2
});

JavaScript's original for...in loop can only get object key names, not key values directly. ES6's for...of loop allows you to traverse and get key values.

var arr = ['a', 'b', 'c', 'd'];

for (let a in arr) {
  console.log(a); // 0 1 2 3
}

for (let a of arr) {
  console.log(a); // a b c d
}

The code above shows that for...in reads key names while for...of reads key values. To get array indices using for...of, you can use the array instance's entries and keys methods (see the "Array Extensions" chapter).

The for...of loop calls the traverser interface, and the array's traverser interface only returns properties with numeric indices. This is another difference from the for...in loop.

let arr = [3, 5, 7];
arr.foo = 'hello';

for (let i in arr) {
  console.log(i); // "0", "1", "2", "foo"
}

for (let i of arr) {
  console.log(i); //  "3", "5", "7"
}

In the code above, for...of does not return array arr's foo property.

Set and Map Structures

Set and Map structures also natively have the Iterator interface and can directly use for...of.

var engines = new Set(["Gecko", "Trident", "Webkit", "Webkit"]);
for (var e of engines) {
  console.log(e);
}
// Gecko
// Trident
// Webkit

var es6 = new Map();
es6.set("edition", 6);
es6.set("committee", "TC39");
es6.set("standard", "ECMA-262");
for (var [name, value] of es6) {
  console.log(name + ": " + value);
}
// edition: 6
// committee: TC39
// standard: ECMA-262

The code above demonstrates traversing Set and Map structures. Two things are worth noting: first, the traversal order follows the order in which members were added to the data structure; second, when traversing a Set, each iteration returns a single value, whereas when traversing a Map, each iteration returns an array whose two members are the current Map member's key and value.

let map = new Map().set('a', 1).set('b', 2);
for (let pair of map) {
  console.log(pair);
}
// ['a', 1]
// ['b', 2]

for (let [key, value] of map) {
  console.log(key + ' : ' + value);
}
// a : 1
// b : 2

Computed Data Structures

Some data structures are computed from existing data structures. For example, ES6 arrays, Sets, and Maps all deploy the following three methods, all of which return traverser objects when called.

• entries() returns a traverser object for traversing arrays of [key name, key value]. For arrays, the key name is the index; for Sets, the key name equals the key value. The Map structure's Iterator interface defaults to calling the entries method.
• keys() returns a traverser object for traversing all key names.
• values() returns a traverser object for traversing all key values.

The traverser objects generated by these three methods all traverse computed data structures.

let arr = ['a', 'b', 'c'];
for (let pair of arr.entries()) {
  console.log(pair);
}
// [0, 'a']
// [1, 'b']
// [2, 'c']

Array-Like Objects

Array-like objects include several types. Here are examples of the for...of loop used with strings, DOM NodeList objects, and arguments objects.

// String
let str = "hello";

for (let s of str) {
  console.log(s); // h e l l o
}

// DOM NodeList object
let paras = document.querySelectorAll("p");

for (let p of paras) {
  p.classList.add("test");
}

// arguments object
function printArgs() {
  for (let x of arguments) {
    console.log(x);
  }
}
printArgs('a', 'b');
// 'a'
// 'b'

For strings, the for...of loop has an additional feature: it correctly recognizes 32-bit UTF-16 characters.

for (let x of 'a\uD83D\uDC0A') {
  console.log(x);
}
// 'a'
// '\uD83D\uDC0A'

Not all array-like objects have the Iterator interface. A simple solution is to use the Array.from method to convert them to arrays.

let arrayLike = { length: 2, 0: 'a', 1: 'b' };

// Error
for (let x of arrayLike) {
  console.log(x);
}

// Correct
for (let x of Array.from(arrayLike)) {
  console.log(x);
}

Objects

For regular objects, the for...of structure cannot be used directly -- it will throw an error. The Iterator interface must be deployed first. However, for...in can still be used to traverse key names.

let es6 = {
  edition: 6,
  committee: "TC39",
  standard: "ECMA-262"
};

for (let e in es6) {
  console.log(e);
}
// edition
// committee
// standard

for (let e of es6) {
  console.log(e);
}
// TypeError: es6[Symbol.iterator] is not a function

The code above shows that for regular objects, for...in can traverse key names, while for...of throws an error.

One solution is to use Object.keys to generate an array of the object's key names, then traverse this array.

for (var key of Object.keys(someObject)) {
  console.log(key + ': ' + someObject[key]);
}

Another approach is to use a Generator function to re-wrap the object.

function* entries(obj) {
  for (let key of Object.keys(obj)) {
    yield [key, obj[key]];
  }
}

for (let [key, value] of entries(obj)) {
  console.log(key, '->', value);
}
// a -> 1
// b -> 2
// c -> 3

Comparison with Other Traversal Syntax

Using arrays as an example, JavaScript provides multiple traversal syntaxes. The most basic is the for loop.

for (var index = 0; index < myArray.length; index++) {
  console.log(myArray[index]);
}

This approach is fairly cumbersome, so arrays provide a built-in forEach method.

myArray.forEach(function (value) {
  console.log(value);
});

The problem with this approach is that you cannot exit the forEach loop midway -- neither break nor return will work.

The for...in loop can traverse the key names of an array.

for (var index in myArray) {
  console.log(myArray[index]);
}

The for...in loop has several drawbacks:

• Array key names are numbers, but for...in traverses them as strings "0", "1", "2", etc.
• for...in traverses not only numeric keys but also manually added keys, and even keys on the prototype chain.
• In some cases, for...in may traverse keys in an arbitrary order.

In summary, for...in is designed primarily for traversing objects and is not suitable for traversing arrays.

The for...of loop has several significant advantages over the approaches above.

for (let value of myArray) {
  console.log(value);
}

• It has the same concise syntax as for...in but without for...in's drawbacks.
• Unlike forEach, it can be used with break, continue, and return.
• It provides a unified operational interface for traversing all data structures.

Here is an example using a break statement to exit a for...of loop.

for (var n of fibonacci) {
  if (n > 1000)
    break;
  console.log(n);
}

The example above outputs Fibonacci sequence terms less than or equal to 1000. If the current term exceeds 1000, the break statement exits the for...of loop.