Generator Function Syntax

Introduction

Basic Concepts

Generator functions are a solution for asynchronous programming provided by ES6, with syntax and behavior completely different from traditional functions. This chapter covers the syntax and API of Generator functions in detail. For their asynchronous programming applications, see the chapter "Asynchronous Applications of Generator Functions". Generator functions can be understood from multiple perspectives. Syntactically, a Generator function can first be understood as a state machine that encapsulates multiple internal states.

Executing a Generator function returns an iterator object. In other words, a Generator function is not only a state machine but also a function that generates iterator objects. The returned iterator object can traverse each internal state of the Generator function one by one.

In terms of form, a Generator function is an ordinary function but has two characteristics. First, there is an asterisk between the function keyword and the function name; second, the function body uses yield expressions internally to define different internal states (the word yield in English means "to produce").

function* helloWorldGenerator() {
  yield 'hello';
  yield 'world';
  return 'ending';
}

var hw = helloWorldGenerator();

The code above defines a Generator function helloWorldGenerator with two yield expressions (hello and world), meaning the function has three states: hello, world, and the return statement (ending execution).

Then, a Generator function is called the same way as an ordinary function — by adding a pair of parentheses after the function name. The difference is that after calling a Generator function, the function does not execute, and what is returned is not the result of the function's execution, but rather a pointer object pointing to internal state, which is the iterator object (Iterator Object) introduced in the previous chapter.

Next, you must call the next method of the iterator object to move the pointer to the next state. In other words, each time the next method is called, the internal pointer starts executing from the beginning of the function or from where it last stopped, until it encounters the next yield expression (or return statement). In short, a Generator function is executed in segments, yield expressions are markers for pausing execution, and the next method can resume execution.

hw.next()
// { value: 'hello', done: false }

hw.next()
// { value: 'world', done: false }

hw.next()
// { value: 'ending', done: true }

hw.next()
// { value: undefined, done: true }

The code above calls the next method a total of four times.

The first call starts executing the Generator function until it encounters the first yield expression. The next method returns an object whose value property is the value of the current yield expression (hello), and whose done property value is false, indicating that the traversal has not yet ended.

The second call resumes the Generator function from where the last yield expression stopped and continues executing until the next yield expression. The value property of the object returned by the next method is the value of the current yield expression (world), and the done property value is false, indicating that the traversal has not yet ended.

The third call resumes the Generator function from where the last yield expression stopped and continues executing until the return statement (if there is no return statement, it runs to the end of the function). The value property of the object returned by the next method is the value of the expression immediately following the return statement (if there is no return statement, the value property is undefined), and the done property value is true, indicating that the traversal has ended.

The fourth call: at this point, the Generator function has already finished running. The next method returns an object with value property undefined and done property true. Any subsequent calls to next will return the same value.

In summary, calling a Generator function returns an iterator object representing the internal pointer of the Generator function. Afterward, each call to the iterator object's next method returns an object with two properties: value and done. The value property represents the value of the current internal state, which is the value of the expression following the yield expression; the done property is a boolean indicating whether the traversal has ended.

ES6 does not specify where the asterisk between the function keyword and the function name should be placed. This means the following notations all work.

function * foo(x, y) { ··· }
function *foo(x, y) { ··· }
function* foo(x, y) { ··· }
function*foo(x, y) { ··· }

Since a Generator function is still an ordinary function, the common convention is the third notation above — the asterisk immediately follows the function keyword. This book also adopts this convention.

yield Expressions

Since the iterator object returned by a Generator function only traverses the next internal state when the next method is called, it effectively provides a function that can be paused. The yield expression serves as the pause marker.

The execution logic of the iterator object's next method is as follows.

(1) When a yield expression is encountered, execution of subsequent operations is paused, and the value of the expression immediately following yield is used as the value property of the returned object.

(2) The next time the next method is called, execution continues until the next yield expression is encountered.

(3) If no more yield expressions are encountered, execution continues to the end of the function until the return statement, and the value of the expression following the return statement is used as the value property of the returned object.

(4) If the function has no return statement, the value property of the returned object is undefined.

It is important to note that the expression after yield is only executed when the next method is called and the internal pointer points to that statement, effectively providing JavaScript with a manual "lazy evaluation" (Lazy Evaluation) syntax feature.

function* gen() {
  yield  123 + 456;
}

In the code above, the expression 123 + 456 after yield is not evaluated immediately. It is only evaluated when the next method moves the pointer to this statement.

The yield expression and the return statement have both similarities and differences. The similarity is that both can return the value of the expression that immediately follows. The difference is that each time yield is encountered, the function pauses execution and continues from that position the next time, whereas the return statement does not have position memory capability. A function can only execute one return statement, but can execute multiple yield expressions. A normal function can only return one value because it can only execute return once; a Generator function can return a series of values because it can have any number of yield expressions. From another perspective, a Generator produces a series of values, which is the origin of its name (in English, "generator" means "something that produces").

A Generator function can be used without yield expressions, in which case it simply becomes a deferred execution function.

function* f() {
  console.log('执行了！')
}

var generator = f();

setTimeout(function () {
  generator.next()
}, 2000);

In the code above, if function f were an ordinary function, it would execute when the variable generator is assigned. However, since function f is a Generator function, it only executes when the next method is called.

Also note that the yield expression can only be used inside Generator functions; using it elsewhere will cause an error.

(function (){
  yield 1;
})()
// SyntaxError: Unexpected number

The code above uses a yield expression inside a regular function, which produces a syntax error.

Here is another example.

var arr = [1, [[2, 3], 4], [5, 6]];

var flat = function* (a) {
  a.forEach(function (item) {
    if (typeof item !== 'number') {
      yield* flat(item);
    } else {
      yield item;
    }
  });
};

for (var f of flat(arr)){
  console.log(f);
}

The code above also produces a syntax error because the parameter of the forEach method is a regular function, but yield expressions are used inside it (this function also uses yield* expressions, which are covered in detail later). One way to fix this is to use a for loop instead.

var arr = [1, [[2, 3], 4], [5, 6]];

var flat = function* (a) {
  var length = a.length;
  for (var i = 0; i < length; i++) {
    var item = a[i];
    if (typeof item !== 'number') {
      yield* flat(item);
    } else {
      yield item;
    }
  }
};

for (var f of flat(arr)) {
  console.log(f);
}
// 1, 2, 3, 4, 5, 6

Also, if a yield expression is used inside another expression, it must be enclosed in parentheses.

function* demo() {
  console.log('Hello' + yield); // SyntaxError
  console.log('Hello' + yield 123); // SyntaxError

  console.log('Hello' + (yield)); // OK
  console.log('Hello' + (yield 123)); // OK
}

When a yield expression is used as a function argument or on the right side of an assignment expression, parentheses are not needed.

function* demo() {
  foo(yield 'a', yield 'b'); // OK
  let input = yield; // OK
}

Relationship with the Iterator Interface

As mentioned in the previous chapter, the Symbol.iterator method of any object is equivalent to that object's iterator generator function. Calling this function returns an iterator object for that object.

Since a Generator function is an iterator generator function, you can assign a Generator to an object's Symbol.iterator property, thereby giving the object an Iterator interface.

var myIterable = {};
myIterable[Symbol.iterator] = function* () {
  yield 1;
  yield 2;
  yield 3;
};

[...myIterable] // [1, 2, 3]

In the code above, a Generator function is assigned to the Symbol.iterator property, giving myIterable an Iterator interface that can now be traversed with the ... operator.

After a Generator function executes, it returns an iterator object. This object itself also has a Symbol.iterator property that, when executed, returns itself.

function* gen(){
  // some code
}

var g = gen();

g[Symbol.iterator]() === g
// true

In the code above, gen is a Generator function and calling it produces an iterator object g. Its Symbol.iterator property is also an iterator generator function that returns itself when executed.

Parameters of the next Method

The yield expression itself has no return value, or rather, it always returns undefined. The next method can take a parameter, and that parameter will be treated as the return value of the previous yield expression.

function* f() {
  for(var i = 0; true; i++) {
    var reset = yield i;
    if(reset) { i = -1; }
  }
}

var g = f();

g.next() // { value: 0, done: false }
g.next() // { value: 1, done: false }
g.next(true) // { value: 0, done: false } // The parameter true is passed to the variable reset above

The code above first defines a Generator function f that can run indefinitely. If the next method has no parameter, each time it runs to the yield expression, the variable reset is always undefined. When the next method takes a parameter true, the variable reset is reset to this parameter (i.e., true), so i becomes -1, and the next loop iteration starts incrementing from -1.

This feature has significant syntactic importance. When a Generator function resumes from a paused state, its context state remains unchanged. Through the next method's parameter, it is possible to inject values into the Generator function body after it has started running. In other words, at different stages of a Generator function's execution, different values can be injected from outside to inside, thereby adjusting function behavior.

Let us look at another example.

function* foo(x) {
  var y = 2 * (yield (x + 1));
  var z = yield (y / 3);
  return (x + y + z);
}

var a = foo(5);
a.next() // Object{value:6, done:false}
a.next() // Object{value:NaN, done:false}
a.next() // Object{value:NaN, done:true}

var b = foo(5);
b.next() // { value:6, done:false }
b.next(12) // { value:8, done:false }
b.next(13) // { value:42, done:true }

In the code above, when the next method is called the second time without a parameter, the value of y equals 2 * undefined (i.e., NaN), and dividing by 3 still yields NaN, so the value property of the returned object is also NaN. When the next method is called the third time without a parameter, z equals undefined, and the value property of the returned object equals 5 + NaN + undefined, which is NaN.

If a parameter is provided to the next method, the result is completely different. In the code above, when the first next method of b is called, it returns the value of x+1, which is 6; when the second next method is called, it sets the previous yield expression's value to 12, so y equals 24, and it returns the value of y / 3, which is 8; when the third next method is called, it sets the previous yield expression's value to 13, so z equals 13. At this point, x equals 5 and y equals 24, so the value of the return statement equals 42.

Note that since the parameter of the next method represents the return value of the previous yield expression, passing a parameter on the first call to next has no effect. The V8 engine directly ignores the parameter on the first call to next; parameters are only effective starting from the second call to next. Semantically, the first next method is used to start the iterator object, so it does not need a parameter.

Here is another example of injecting a value into a Generator function body through the next method's parameter.

function* dataConsumer() {
  console.log('Started');
  console.log(`1. ${yield}`);
  console.log(`2. ${yield}`);
  return 'result';
}

let genObj = dataConsumer();
genObj.next();
// Started
genObj.next('a')
// 1. a
genObj.next('b')
// 2. b

The code above is a very intuitive example: each time a value is input to the Generator function through the next method, it is printed out.

If you want to input a value on the very first call to next, you can wrap the Generator function with an additional layer.

function wrapper(generatorFunction) {
  return function (...args) {
    let generatorObject = generatorFunction(...args);
    generatorObject.next();
    return generatorObject;
  };
}

const wrapped = wrapper(function* () {
  console.log(`First input: ${yield}`);
  return 'DONE';
});

wrapped().next('hello!')
// First input: hello!

In the code above, without first wrapping the Generator function with wrapper, it would be impossible to input a parameter on the first call to next.

for...of Loop

The for...of loop can automatically traverse the Iterator object generated when a Generator function runs, eliminating the need to call the next method.

function* foo() {
  yield 1;
  yield 2;
  yield 3;
  yield 4;
  yield 5;
  return 6;
}

for (let v of foo()) {
  console.log(v);
}
// 1 2 3 4 5

The code above uses a for...of loop to display the values of five yield expressions in sequence. Note that once the done property of the object returned by the next method is true, the for...of loop terminates and does not include that return object. Therefore, the 6 returned by the return statement is not included in the for...of loop.

Below is an example of implementing the Fibonacci sequence using a Generator function and for...of loop.

function* fibonacci() {
  let [prev, curr] = [0, 1];
  for (;;) {
    yield curr;
    [prev, curr] = [curr, prev + curr];
  }
}

for (let n of fibonacci()) {
  if (n > 1000) break;
  console.log(n);
}

As can be seen from the code above, when using the for...of statement, there is no need to use the next method.

Using the for...of loop, you can write a method to traverse any object. Native JavaScript objects do not have a traversal interface and cannot use the for...of loop. By adding this interface through a Generator function, they can then be used.

function* objectEntries(obj) {
  let propKeys = Reflect.ownKeys(obj);

  for (let propKey of propKeys) {
    yield [propKey, obj[propKey]];
  }
}

let jane = { first: 'Jane', last: 'Doe' };

for (let [key, value] of objectEntries(jane)) {
  console.log(`${key}: ${value}`);
}
// first: Jane
// last: Doe

In the code above, the object jane does not natively have an Iterator interface and cannot be traversed with for...of. Here, we use the Generator function objectEntries to add an iterator interface to it, making for...of traversal possible. Another way to add an iterator interface is to assign the Generator function to the object's Symbol.iterator property.

function* objectEntries() {
  let propKeys = Object.keys(this);

  for (let propKey of propKeys) {
    yield [propKey, this[propKey]];
  }
}

let jane = { first: 'Jane', last: 'Doe' };

jane[Symbol.iterator] = objectEntries;

for (let [key, value] of jane) {
  console.log(`${key}: ${value}`);
}
// first: Jane
// last: Doe

Besides the for...of loop, the spread operator (...), destructuring assignment, and the Array.from method all internally call the iterator interface. This means they can all take the Iterator object returned by a Generator function as a parameter.

function* numbers () {
  yield 1
  yield 2
  return 3
  yield 4
}

// Spread operator
[...numbers()] // [1, 2]

// Array.from method
Array.from(numbers()) // [1, 2]

// Destructuring assignment
let [x, y] = numbers();
x // 1
y // 2

// for...of loop
for (let n of numbers()) {
  console.log(n)
}
// 1
// 2

Generator.prototype.throw()

The iterator object returned by a Generator function has a throw method that can throw an error outside the function body, which is then caught inside the Generator function body.

var g = function* () {
  try {
    yield;
  } catch (e) {
    console.log('internal catch', e);
  }
};

var i = g();
i.next();

try {
  i.throw('a');
  i.throw('b');
} catch (e) {
  console.log('external catch', e);
}
// internal catch a
// external catch b

In the code above, the iterator object i throws two errors consecutively. The first error is caught by the catch statement inside the Generator function body. When i throws the second error, since the internal catch statement has already been executed and will not catch this error again, the error is thrown outside the Generator function body and caught by the external catch statement.

The throw method can accept a parameter, which will be received by the catch statement. It is recommended to throw instances of Error objects.

var g = function* () {
  try {
    yield;
  } catch (e) {
    console.log(e);
  }
};

var i = g();
i.next();
i.throw(new Error('出错了！'));
// Error: 出错了！(…)

Be careful not to confuse the iterator object's throw method with the global throw command. In the code above, the error is thrown using the iterator object's throw method, not the throw command. The latter can only be caught by a catch statement outside the function body.

var g = function* () {
  while (true) {
    try {
      yield;
    } catch (e) {
      if (e != 'a') throw e;
      console.log('internal catch', e);
    }
  }
};

var i = g();
i.next();

try {
  throw new Error('a');
  throw new Error('b');
} catch (e) {
  console.log('external catch', e);
}
// external catch [Error: a]

The reason the code above only catches a is that after the catch block outside the function body catches the thrown a error, it does not continue executing the remaining statements inside the try block.

If the Generator function body does not have a try...catch block deployed, the error thrown by the throw method will be caught by the external try...catch block.

var g = function* () {
  while (true) {
    yield;
    console.log('internal catch', e);
  }
};

var i = g();
i.next();

try {
  i.throw('a');
  i.throw('b');
} catch (e) {
  console.log('external catch', e);
}
// external catch a

In the code above, the Generator function g does not have a try...catch block deployed internally, so the thrown error is directly caught by the external catch block.

If neither inside nor outside the Generator function has a try...catch block deployed, the program will throw an error and terminate execution.

var gen = function* gen(){
  yield console.log('hello');
  yield console.log('world');
}

var g = gen();
g.next();
g.throw();
// hello
// Uncaught undefined

In the code above, after g.throw throws an error, no try...catch block can catch it, causing the program to error and terminate execution.

For an error thrown by the throw method to be caught internally, the next method must have been called at least once.

function* gen() {
  try {
    yield 1;
  } catch (e) {
    console.log('internal catch');
  }
}

var g = gen();
g.throw(1);
// Uncaught 1

In the code above, when g.throw(1) is executed, the next method has never been called. At this point, the thrown error will not be caught internally but will be thrown externally, causing a program error. This behavior makes sense because calling the next method for the first time is equivalent to starting the execution of the Generator function's internal code. Otherwise, the Generator function has not yet started executing, so a throw method error can only be thrown outside the function.

After the throw method is caught, it will additionally execute the next yield expression. In other words, it will additionally perform one next method call.

var gen = function* gen(){
  try {
    yield console.log('a');
  } catch (e) {
    // ...
  }
  yield console.log('b');
  yield console.log('c');
}

var g = gen();
g.next() // a
g.throw() // b
g.next() // c

In the code above, after the g.throw method is caught, it automatically executes one next method call, which is why b is printed. Also, as long as a try...catch block is deployed inside the Generator function, the error thrown by the iterator's throw method does not affect the next traversal.

Additionally, the throw command and the g.throw method are unrelated; they do not affect each other.

var gen = function* gen(){
  yield console.log('hello');
  yield console.log('world');
}

var g = gen();
g.next();

try {
  throw new Error();
} catch (e) {
  g.next();
}
// hello
// world

In the code above, the error thrown by the throw command does not affect the state of the iterator, so both next method calls perform correct operations.

This mechanism of catching errors inside the function body greatly facilitates error handling. Multiple yield expressions can use just one try...catch block to catch errors. With the callback function approach, catching multiple errors would require writing an error handling statement inside each function. Now, only one catch statement needs to be written inside the Generator function.

Errors thrown outside the Generator function body can be caught inside the function body; conversely, errors thrown inside the Generator function body can also be caught by an external catch.

function* foo() {
  var x = yield 3;
  var y = x.toUpperCase();
  yield y;
}

var it = foo();

it.next(); // { value:3, done:false }

try {
  it.next(42);
} catch (err) {
  console.log(err);
}

In the code above, the second next method passes the parameter 42 into the function body. Since a number does not have a toUpperCase method, it throws a TypeError error, which is caught by the external catch.

Once an error is thrown during the execution of a Generator and is not caught internally, the Generator will not continue executing. If next is called after that, it will return an object with the value property equal to undefined and the done property equal to true, meaning the JavaScript engine considers this Generator to have finished running.

function* g() {
  yield 1;
  console.log('throwing an exception');
  throw new Error('generator broke!');
  yield 2;
  yield 3;
}

function log(generator) {
  var v;
  console.log('starting generator');
  try {
    v = generator.next();
    console.log('first next method run', v);
  } catch (err) {
    console.log('caught error', v);
  }
  try {
    v = generator.next();
    console.log('second next method run', v);
  } catch (err) {
    console.log('caught error', v);
  }
  try {
    v = generator.next();
    console.log('third next method run', v);
  } catch (err) {
    console.log('caught error', v);
  }
  console.log('caller done');
}

log(g());
// starting generator
// first next method run { value: 1, done: false }
// throwing an exception
// caught error { value: 1, done: false }
// third next method run { value: undefined, done: true }
// caller done

The code above runs the next method three times in total. On the second run, an error is thrown, and by the third run, the Generator function has already finished and will not continue executing.

Generator.prototype.return()

The iterator object returned by a Generator function also has a return method that can return a given value and terminate traversal of the Generator function.

function* gen() {
  yield 1;
  yield 2;
  yield 3;
}

var g = gen();

g.next()        // { value: 1, done: false }
g.return('foo') // { value: "foo", done: true }
g.next()        // { value: undefined, done: true }

In the code above, after the iterator object g calls the return method, the value property of the return value is the parameter foo of the return method. The traversal of the Generator function is then terminated; the done property of the return value is true, and any subsequent calls to next will always return done as true.

If the return method is called without a parameter, the value property of the return value is undefined.

function* gen() {
  yield 1;
  yield 2;
  yield 3;
}

var g = gen();

g.next()        // { value: 1, done: false }
g.return() // { value: undefined, done: true }

If the Generator function body has a try...finally block and the try block is currently being executed, the return method will cause immediate entry into the finally block. After the finally block finishes executing, the entire function ends.

function* numbers () {
  yield 1;
  try {
    yield 2;
    yield 3;
  } finally {
    yield 4;
    yield 5;
  }
  yield 6;
}
var g = numbers();
g.next() // { value: 1, done: false }
g.next() // { value: 2, done: false }
g.return(7) // { value: 4, done: false }
g.next() // { value: 5, done: false }
g.next() // { value: 7, done: true }

In the code above, after calling the return() method, the finally block begins executing and the remaining code in try is not executed. After the finally block finishes, the return value specified by the return() method is returned.

Commonalities of next(), throw(), and return()

The three methods next(), throw(), and return() are essentially the same thing and can be understood together. Their purpose is to resume execution of the Generator function and replace the yield expression with different statements.

next() replaces the yield expression with a value.

const g = function* (x, y) {
  let result = yield x + y;
  return result;
};

const gen = g(1, 2);
gen.next(); // Object {value: 3, done: false}

gen.next(1); // Object {value: 1, done: true}
// Equivalent to replacing let result = yield x + y
// with let result = 1;

In the code above, the second next(1) method call is equivalent to replacing the yield expression with a value 1. If the next method has no parameter, it is equivalent to replacing with undefined.

throw() replaces the yield expression with a throw statement.

gen.throw(new Error('出错了')); // Uncaught Error: 出错了
// Equivalent to replacing let result = yield x + y
// with let result = throw(new Error('出错了'));

return() replaces the yield expression with a return statement.

gen.return(2); // Object {value: 2, done: true}
// Equivalent to replacing let result = yield x + y
// with let result = return 2;

yield* Expression

If inside a Generator function you want to call another Generator function, you need to manually complete the traversal inside the former function body.

function* foo() {
  yield 'a';
  yield 'b';
}

function* bar() {
  yield 'x';
  // Manually traverse foo()
  for (let i of foo()) {
    console.log(i);
  }
  yield 'y';
}

for (let v of bar()){
  console.log(v);
}
// x
// a
// b
// y

In the code above, both foo and bar are Generator functions. Calling foo inside bar requires manual traversal of foo. If there are multiple nested Generator functions, the code becomes very tedious.

ES6 provides the yield* expression as a solution, used to execute another Generator function inside a Generator function.

function* bar() {
  yield 'x';
  yield* foo();
  yield 'y';
}

// Equivalent to
function* bar() {
  yield 'x';
  yield 'a';
  yield 'b';
  yield 'y';
}

// Equivalent to
function* bar() {
  yield 'x';
  for (let v of foo()) {
    yield v;
  }
  yield 'y';
}

for (let v of bar()){
  console.log(v);
}
// "x"
// "a"
// "b"
// "y"

Let us look at another comparison example.

function* inner() {
  yield 'hello!';
}

function* outer1() {
  yield 'open';
  yield inner();
  yield 'close';
}

var gen = outer1()
gen.next().value // "open"
gen.next().value // Returns an iterator object
gen.next().value // "close"

function* outer2() {
  yield 'open'
  yield* inner()
  yield 'close'
}

var gen = outer2()
gen.next().value // "open"
gen.next().value // "hello!"
gen.next().value // "close"

In the example above, outer2 uses yield* while outer1 does not. The result is that outer1 returns an iterator object, while outer2 returns the internal value of that iterator object.

From a syntactic perspective, if the yield expression is followed by an iterator object, an asterisk needs to be added after the yield expression to indicate that it returns an iterator object. This is called a yield* expression.

let delegatedIterator = (function* () {
  yield 'Hello!';
  yield 'Bye!';
}());

let delegatingIterator = (function* () {
  yield 'Greetings!';
  yield* delegatedIterator;
  yield 'Ok, bye.';
}());

for(let value of delegatingIterator) {
  console.log(value);
}
// "Greetings!
// "Hello!"
// "Bye!"
// "Ok, bye."

In the code above, delegatingIterator is the delegator and delegatedIterator is the delegate. Since the yield* delegatedIterator statement produces an iterator, the asterisk is needed. The result is that one iterator traverses multiple Generator functions, creating a recursive effect.

The Generator function following yield* (without a return statement) is equivalent to deploying a for...of loop inside the Generator function.

function* concat(iter1, iter2) {
  yield* iter1;
  yield* iter2;
}

// Equivalent to

function* concat(iter1, iter2) {
  for (var value of iter1) {
    yield value;
  }
  for (var value of iter2) {
    yield value;
  }
}

The code above shows that the Generator function following yield* (without a return statement) is simply a shorthand for for...of and can be completely replaced by the latter. Conversely, when there is a return statement, the form var value = yield* iterator is needed to obtain the value of the return statement.

If yield* is followed by an array, since arrays natively support iterators, it will traverse the array members.

function* gen(){
  yield* ["a", "b", "c"];
}

gen().next() // { value:"a", done:false }

In the code above, if no asterisk follows the yield command, it returns the entire array; with the asterisk, it returns the array's iterator object.

In fact, any data structure with an Iterator interface can be traversed by yield*.

let read = (function* () {
  yield 'hello';
  yield* 'hello';
})();

read.next().value // "hello"
read.next().value // "h"

In the code above, the yield expression returns the entire string, while the yield* statement returns individual characters. Because strings have an Iterator interface, they are traversed by yield*.

If the delegated Generator function has a return statement, then it can return data to the delegating Generator function.

function* foo() {
  yield 2;
  yield 3;
  return "foo";
}

function* bar() {
  yield 1;
  var v = yield* foo();
  console.log("v: " + v);
  yield 4;
}

var it = bar();

it.next()
// {value: 1, done: false}
it.next()
// {value: 2, done: false}
it.next()
// {value: 3, done: false}
it.next();
// "v: foo"
// {value: 4, done: false}
it.next()
// {value: undefined, done: true}

In the code above, on the fourth call to the next method, output appears on the screen because the return statement of function foo provides a return value to function bar.

Let us look at another example.

function* genFuncWithReturn() {
  yield 'a';
  yield 'b';
  return 'The result';
}
function* logReturned(genObj) {
  let result = yield* genObj;
  console.log(result);
}

[...logReturned(genFuncWithReturn())]
// The result
// Value is [ 'a', 'b' ]

In the code above, there are two traversals. The first is the spread operator traversing the iterator object returned by logReturned, and the second is the yield* statement traversing the iterator object returned by genFuncWithReturn. The effects of these two traversals are additive, ultimately appearing as the spread operator traversing the iterator object returned by genFuncWithReturn. Therefore, the final data expression evaluates to [ 'a', 'b' ]. However, the return value The result of genFuncWithReturn's return statement is returned to the internal result variable of logReturned, producing terminal output.

The yield* command can conveniently extract all members of a nested array.

function* iterTree(tree) {
  if (Array.isArray(tree)) {
    for(let i=0; i < tree.length; i++) {
      yield* iterTree(tree[i]);
    }
  } else {
    yield tree;
  }
}

const tree = [ 'a', ['b', 'c'], ['d', 'e'] ];

for(let x of iterTree(tree)) {
  console.log(x);
}
// a
// b
// c
// d
// e

Since the spread operator ... calls the Iterator interface by default, the function above can also be used for flattening nested arrays.

[...iterTree(tree)] // ["a", "b", "c", "d", "e"]

Below is a slightly more complex example using the yield* statement to traverse a complete binary tree.

// Below is the constructor for a binary tree,
// with three parameters: left subtree, current node, and right subtree
function Tree(left, label, right) {
  this.left = left;
  this.label = label;
  this.right = right;
}

// Below is the inorder traversal function.
// Since it returns an iterator, a generator function is used.
// The function body uses a recursive algorithm, so yield* is used to traverse the left and right subtrees
function* inorder(t) {
  if (t) {
    yield* inorder(t.left);
    yield t.label;
    yield* inorder(t.right);
  }
}

// Below generates the binary tree
function make(array) {
  // Check if it is a leaf node
  if (array.length == 1) return new Tree(null, array[0], null);
  return new Tree(make(array[0]), array[1], make(array[2]));
}
let tree = make([[['a'], 'b', ['c']], 'd', [['e'], 'f', ['g']]]);

// Traverse the binary tree
var result = [];
for (let node of inorder(tree)) {
  result.push(node);
}

result
// ['a', 'b', 'c', 'd', 'e', 'f', 'g']

Generator Functions as Object Properties

If an object's property is a Generator function, it can be abbreviated as follows.

let obj = {
  * myGeneratorMethod() {
    ···
  }
};

In the code above, the asterisk before the myGeneratorMethod property indicates that this property is a Generator function.

Its complete form is as follows, which is equivalent to the notation above.

let obj = {
  myGeneratorMethod: function* () {
    // ···
  }
};

The this of Generator Functions

A Generator function always returns an iterator. ES6 specifies that this iterator is an instance of the Generator function and inherits methods on the Generator function's prototype object.

function* g() {}

g.prototype.hello = function () {
  return 'hi!';
};

let obj = g();

obj instanceof g // true
obj.hello() // 'hi!'

The code above shows that the iterator obj returned by Generator function g is an instance of g and inherits from g.prototype. However, if g is treated as a regular constructor function, it will not work because g always returns an iterator object, not the this object.

function* g() {
  this.a = 11;
}

let obj = g();
obj.next();
obj.a // undefined

In the code above, Generator function g adds a property a to the this object, but the obj object cannot access this property.

Generator functions also cannot be used with the new command and will throw an error.

function* F() {
  yield this.x = 2;
  yield this.y = 3;
}

new F()
// TypeError: F is not a constructor

In the code above, using the new command with constructor F causes an error because F is not a constructor.

So, is there a way to make a Generator function return a normal object instance that can use the next method while also accessing normal this properties?

Here is a workaround. First, generate an empty object and use the call method to bind this inside the Generator function. This way, after the constructor is called, this empty object becomes the instance object of the Generator function.

function* F() {
  this.a = 1;
  yield this.b = 2;
  yield this.c = 3;
}
var obj = {};
var f = F.call(obj);

f.next();  // Object {value: 2, done: false}
f.next();  // Object {value: 3, done: false}
f.next();  // Object {value: undefined, done: true}

obj.a // 1
obj.b // 2
obj.c // 3

In the code above, first the this object inside F is bound to obj, then it is called, returning an Iterator object. This object executes the next method three times (because F has two yield expressions), completing all the code inside F. At this point, all internal properties are bound to the obj object, so obj becomes an instance of F.

In the code above, the iterator object f is what is executed, but the generated object instance is obj. Is there a way to unify these two objects?

One approach is to replace obj with F.prototype.

function* F() {
  this.a = 1;
  yield this.b = 2;
  yield this.c = 3;
}
var f = F.call(F.prototype);

f.next();  // Object {value: 2, done: false}
f.next();  // Object {value: 3, done: false}
f.next();  // Object {value: undefined, done: true}

f.a // 1
f.b // 2
f.c // 3

Then, by changing F into a constructor, you can use the new command on it.

function* gen() {
  this.a = 1;
  yield this.b = 2;
  yield this.c = 3;
}

function F() {
  return gen.call(gen.prototype);
}

var f = new F();

f.next();  // Object {value: 2, done: false}
f.next();  // Object {value: 3, done: false}
f.next();  // Object {value: undefined, done: true}

f.a // 1
f.b // 2
f.c // 3

Meaning

Generator and State Machines

Generator is the best structure for implementing state machines. For example, the clock function below is a state machine.

var ticking = true;
var clock = function() {
  if (ticking)
    console.log('Tick!');
  else
    console.log('Tock!');
  ticking = !ticking;
}

The clock function above has two states (Tick and Tock), and each run changes the state once. If implemented using a Generator, it looks like this.

var clock = function* () {
  while (true) {
    console.log('Tick!');
    yield;
    console.log('Tock!');
    yield;
  }
};

Comparing the Generator implementation with the ES5 implementation above, we can see that it eliminates the external variable ticking used to save state, making it more concise, safer (the state cannot be tampered with illegally), more in line with functional programming principles, and more elegant in notation. The reason a Generator can avoid using an external variable to save state is that it inherently contains state information, namely whether it is currently in a paused state.

Generator and Coroutines

A coroutine is a way of running programs that can be understood as "cooperative threads" or "cooperative functions". Coroutines can be implemented with either single threading or multi-threading. The former is a special kind of subroutine, and the latter is a special kind of thread.

(1) Differences between coroutines and subroutines

Traditional "subroutines" use a stack-based "last-in, first-out" execution model, where the parent function only ends execution after the called subfunction has completely finished. Coroutines are different: multiple threads (in single-threaded scenarios, meaning multiple functions) can execute in parallel, but only one thread (or function) is in the running state while the others are in a suspended state. Threads (or functions) can exchange execution control. In other words, a thread (or function) can pause in the middle of execution, hand execution control to another thread (or function), and resume execution when it later regains control. These threads (or functions) that can execute in parallel and exchange execution control are called coroutines.

In terms of implementation, in memory, subroutines only use one stack, while coroutines simultaneously maintain multiple stacks with only one stack in the running state. In other words, coroutines achieve parallel multitasking at the cost of additional memory usage.

(2) Differences between coroutines and regular threads

It is easy to see that coroutines are suited for multitasking environments. In this sense, they are similar to regular threads — both have their own execution context and can share global variables. The difference is that multiple regular threads can be in the running state simultaneously, but only one coroutine can be running while others are in a paused state. Additionally, regular threads are preemptive — which thread gets priority access to resources is decided by the runtime environment — but coroutines are cooperative, and execution control is distributed by the coroutines themselves.

Since JavaScript is a single-threaded language that can only maintain one call stack, introducing coroutines allows each task to maintain its own call stack. The biggest benefit is that when an error is thrown, the original call stack can be found, unlike asynchronous operation callbacks where, once an error occurs, the original call stack has long since ended.

Generator functions are ES6's implementation of coroutines, but they are an incomplete implementation. Generator functions are called "semi-coroutines", meaning that only the caller of a Generator function can return execution control to the Generator function. In a fully implemented coroutine, any function can make a paused coroutine resume execution.

If you treat Generator functions as coroutines, you can write multiple tasks that need to cooperate as Generator functions, using yield expressions to exchange control among them.

Generator and Context

When JavaScript code runs, it produces a global context environment (also called runtime environment) that contains all current variables and objects. Then, when executing a function (or block-level code), a function execution context is created on top of the current context, becoming the current (active) context, forming a context stack.

This stack is a "last-in, first-out" data structure where the most recently created context finishes executing first and exits the stack, then its lower context finishes executing, and so on until all code has finished executing and the stack is cleared.

Generator functions do not work this way. The context created by their execution, upon encountering a yield command, temporarily exits the stack but does not disappear — all variables and objects are frozen in their current state. When the next command is executed on it, this context rejoins the call stack and the frozen variables and objects resume execution.

function* gen() {
  yield 1;
  return 2;
}

let g = gen();

console.log(
  g.next().value,
  g.next().value,
);

In the code above, when g.next() is executed the first time, the Generator function gen's context is added to the stack and begins running gen's internal code. When yield 1 is encountered, gen's context exits the stack and internal state is frozen. When g.next() is executed the second time, gen's context rejoins the stack as the current context and resumes execution.

Applications

Generator can pause function execution and return values of arbitrary expressions. This characteristic gives Generator multiple application scenarios.

(1) Synchronous Expression of Asynchronous Operations

The pause-execution effect of Generator functions means that asynchronous operations can be placed inside yield expressions and then executed when the next method is called. This effectively eliminates the need to write callback functions, because subsequent operations of the asynchronous operation can be placed after the yield expression — they will wait until the next method is called to execute. Therefore, an important practical significance of Generator functions is handling asynchronous operations and rewriting callback functions.

function* loadUI() {
  showLoadingScreen();
  yield loadUIDataAsynchronously();
  hideLoadingScreen();
}
var loader = loadUI();
// Load UI
loader.next()

// Unload UI
loader.next()

In the code above, when the loadUI function is called the first time, the function does not execute and only returns an iterator. The next time next is called on the iterator, the Loading screen is displayed (showLoadingScreen), and data is loaded asynchronously (loadUIDataAsynchronously). When the data has finished loading, using next again will hide the Loading screen. As can be seen, the benefit of this approach is that all Loading screen logic is encapsulated in a single function, proceeding step by step very clearly.

Ajax is a typical asynchronous operation. By deploying Ajax operations through Generator functions, they can be expressed synchronously.

function* main() {
  var result = yield request("http://some.url");
  var resp = JSON.parse(result);
    console.log(resp.value);
}

function request(url) {
  makeAjaxCall(url, function(response){
    it.next(response);
  });
}

var it = main();
it.next();

The main function in the code above retrieves data through an Ajax operation. As can be seen, aside from the extra yield, it is almost identical to synchronous code. Note that the next method in the makeAjaxCall function must include the response parameter, because the yield expression itself has no value and always equals undefined.

Below is another example of reading a text file line by line through a Generator function.

function* numbers() {
  let file = new FileReader("numbers.txt");
  try {
    while(!file.eof) {
      yield parseInt(file.readLine(), 10);
    }
  } finally {
    file.close();
  }
}

The code above opens a text file and uses the yield expression to manually read the file line by line.

(2) Control Flow Management

If a multi-step operation is very time-consuming, using callbacks might look like this.

step1(function (value1) {
  step2(value1, function(value2) {
    step3(value2, function(value3) {
      step4(value3, function(value4) {
        // Do something with value4
      });
    });
  });
});

Using Promise to rewrite the code above.

Promise.resolve(step1)
  .then(step2)
  .then(step3)
  .then(step4)
  .then(function (value4) {
    // Do something with value4
  }, function (error) {
    // Handle any error from step1 through step4
  })
  .done();

The code above has already transformed the callbacks into linear execution form, but introduces a lot of Promise syntax. Generator functions can further improve the code execution flow.

function* longRunningTask(value1) {
  try {
    var value2 = yield step1(value1);
    var value3 = yield step2(value2);
    var value4 = yield step3(value3);
    var value5 = yield step4(value4);
    // Do something with value4
  } catch (e) {
    // Handle any error from step1 through step4
  }
}

Then, use a function to automatically execute all steps in order.

scheduler(longRunningTask(initialValue));

function scheduler(task) {
  var taskObj = task.next(task.value);
  // If the Generator function has not ended, keep calling
  if (!taskObj.done) {
    task.value = taskObj.value
    scheduler(task);
  }
}

Note that the approach above is only suitable for synchronous operations, meaning all tasks must be synchronous and cannot have asynchronous operations. This is because the code continues executing as soon as it gets a return value, without checking when asynchronous operations complete. For controlling asynchronous operation flows, see the "Asynchronous Operations" chapter later.

Below, using the feature that for...of loops automatically execute yield commands in sequence, a more general method of control flow management is provided.

let steps = [step1Func, step2Func, step3Func];

function* iterateSteps(steps){
  for (var i=0; i< steps.length; i++){
    var step = steps[i];
    yield step();
  }
}

In the code above, the array steps wraps the multiple steps of a task, and the Generator function iterateSteps adds a yield command to each of these steps in sequence.

After breaking a task into steps, a project can also be broken into multiple tasks that execute sequentially.

let jobs = [job1, job2, job3];

function* iterateJobs(jobs){
  for (var i=0; i< jobs.length; i++){
    var job = jobs[i];
    yield* iterateSteps(job.steps);
  }
}

In the code above, the array jobs wraps multiple tasks of a project, and the Generator function iterateJobs adds a yield* command to each of these tasks in sequence.

Finally, a for...of loop can be used to execute all steps of all tasks at once.

for (var step of iterateJobs(jobs)){
  console.log(step.id);
}

Again, the approach above can only be used when all steps are synchronous operations and cannot have asynchronous steps. If you want to execute asynchronous steps sequentially, you must use the methods introduced in the "Asynchronous Operations" chapter.

The essence of for...of is a while loop, so the code above actually executes the following logic.

var it = iterateJobs(jobs);
var res = it.next();

while (!res.done){
  var result = res.value;
  // ...
  res = it.next();
}

(3) Deploying the Iterator Interface

Using Generator functions, the Iterator interface can be deployed on any object.

function* iterEntries(obj) {
  let keys = Object.keys(obj);
  for (let i=0; i < keys.length; i++) {
    let key = keys[i];
    yield [key, obj[key]];
  }
}

let myObj = { foo: 3, bar: 7 };

for (let [key, value] of iterEntries(myObj)) {
  console.log(key, value);
}

// foo 3
// bar 7

In the code above, myObj is an ordinary object. Through the iterEntries function, it now has an Iterator interface. In other words, the next method can be deployed on any object.

Below is an example of deploying the Iterator interface on an array, even though arrays natively have this interface.

function* makeSimpleGenerator(array){
  var nextIndex = 0;

  while(nextIndex < array.length){
    yield array[nextIndex++];
  }
}

var gen = makeSimpleGenerator(['yo', 'ya']);

gen.next().value // 'yo'
gen.next().value // 'ya'
gen.next().done  // true

(4) As a Data Structure

Generator can be viewed as a data structure — more precisely, as an array structure, because a Generator function can return a series of values, meaning it can provide an array-like interface for any expression.

function* doStuff() {
  yield fs.readFile.bind(null, 'hello.txt');
  yield fs.readFile.bind(null, 'world.txt');
  yield fs.readFile.bind(null, 'and-such.txt');
}

The code above returns three functions in sequence, but because a Generator function is used, these three returned functions can be handled as if they were an array.

for (task of doStuff()) {
  // task is a function that can be used like a callback function
}

In fact, expressed in ES5, an array can completely simulate this usage of Generator.

function doStuff() {
  return [
    fs.readFile.bind(null, 'hello.txt'),
    fs.readFile.bind(null, 'world.txt'),
    fs.readFile.bind(null, 'and-such.txt')
  ];
}

The function above can be processed with the exact same for...of loop! Comparing the two, it is not hard to see that Generator gives data or operations an array-like interface.