{"action":"created","issue":{"url":"https://api.github.com/repos/Microsoft/TypeScript/issues/2873","labels_url":"https://api.github.com/repos/Microsoft/TypeScript/issues/2873/labels{/name}","comments_url":"https://api.github.com/repos/Microsoft/TypeScript/issues/2873/comments","events_url":"https://api.github.com/repos/Microsoft/TypeScript/issues/2873/events","html_url":"https://github.com/Microsoft/TypeScript/issues/2873","id":70221605,"number":2873,"title":"Proposal for generators design","user":{"login":"JsonFreeman","id":8052525,"avatar_url":"https://avatars.githubusercontent.com/u/8052525?v=3","gravatar_id":"","url":"https://api.github.com/users/JsonFreeman","html_url":"https://github.com/JsonFreeman","followers_url":"https://api.github.com/users/JsonFreeman/followers","following_url":"https://api.github.com/users/JsonFreeman/following{/other_user}","gists_url":"https://api.github.com/users/JsonFreeman/gists{/gist_id}","starred_url":"https://api.github.com/users/JsonFreeman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JsonFreeman/subscriptions","organizations_url":"https://api.github.com/users/JsonFreeman/orgs","repos_url":"https://api.github.com/users/JsonFreeman/repos","events_url":"https://api.github.com/users/JsonFreeman/events{/privacy}","received_events_url":"https://api.github.com/users/JsonFreeman/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Microsoft/TypeScript/labels/ES6","name":"ES6","color":"c7def8"},{"url":"https://api.github.com/repos/Microsoft/TypeScript/labels/Spec","name":"Spec","color":"207de5"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":66,"created_at":"2015-04-22T20:02:06Z","updated_at":"2015-04-30T08:21:42Z","closed_at":null,"body":"A generator is a syntactic way to declare a function that can yield. Yielding will give a value to the caller of the next() method of the generator, and will suspend execution at the yield point. A generator also supports `yield *` which means that it will delegate to another generator and yield the results that the inner generator yields. `yield` and `yield *` are also bi-directional. A value can flow in as well as out.\r\n\r\nLike an iterator, the thing returned by the next method has a done property and a value property. Yielding sets done to false, and returning sets done to true.\r\n\r\nA generator is also iterable. You can iterate over the yielded values of the generator, using for-of, spread or array destructuring. However, only yielded values come out when you use a generator in this way. Returned values are never exposed. As a result, this proposal only considers the value type of next() when the done property is false, since those are the ones that will normally be observed.\r\n\r\n# Basic support for generators\r\n\r\n## Type annotation on a generator\r\nA generator function can have a return type annotation, just like a function. The annotation represents the type of the generator returned by the function. Here is an example:\r\n```ts\r\nfunction *g(): Iterable<string> {\r\n    for (var i = 0; i < 100; i++) {\r\n        yield \"\"; // string is assignable to string\r\n    }\r\n    yield * otherStringGenerator(); // otherStringGenerator must be iterable and element type assignable to string\r\n}\r\n```\r\nHere are the rules:\r\n\r\n* ~~The type annotation must be assignable to `Iterable<any>`~~.\r\n    * This has been revised: `IterableIterator<any>` must be assignable to the type annotation instead.\r\n* The operand of every yield expression (if present) must be assignable to the element type of the generator (string in this case)\r\n* The operand of every `yield *` expression must be assignable to `Iterable<any>`\r\n* The element type of the operand of every `yield *` expression must be assignable to the element type of the generator. (string is assignable to string)\r\n* The operand of a `yield` (if present) expression is contextually typed by the element type of the generator (string)\r\n* The operand of a `yield *` expression is contextually typed by the type of the generator (`Iterable<string>`)\r\n* A `yield` expression has type any.\r\n* A `yield *` expression has type any.\r\n* ~~The generator is allowed to have return expressions as well, but they are ignored for the purposes of type checking the generator type.~~ The generator cannot have return expressions\r\n\r\n* Open question: Do we want to give an error for a return expression that is not assignable to the element type? If so, we would also contextually type it by the element type.\r\n    * Answer: we will give an error on all return expressions in a generator. Consider relaxing this later.\r\n* Open question: Should we allow void generators?\r\n    * Answer: no\r\n\r\n## Inferring the type of a generator\r\nA generator function with no type annotation can have the type annotation inferred. So in the following case, the type will be inferred from the yield statements:\r\n```ts\r\nfunction *g() {\r\n    for (var i = 0; i < 100; i++) {\r\n        yield \"\"; // infer string\r\n    }\r\n    yield * otherStringGenerator(); // infer element type of otherStringGenerator\r\n}\r\n```\r\n* Rather than inferring Iterable, we will infer IterableIterator, with some element type. The reason is that someone can call next directly on the generator without first getting its iterator. A generator is in fact an iterator as well as an iterable.\r\n* The element type is the common supertype of all the yield operands and the element types of all the `yield *` operands.\r\n* It is an error if there is no common supertype.\r\n* As before, the operand of every `yield *` expression must be assignable to `Iterable<any>`\r\n* `yield` and `yield *` expressions again have type any\r\n* If the generator is contextually typed, the operands of `yield` expressions are contextually typed by the element type of the contextual type\r\n* If the generator is contextually typed, the operands of `yield *` expressions are contextually typed by the contextual type.\r\n* ~~Again, return expressions are allowed, but not used for inferring the element type.~~ Return expressions are not allowed. Consider relaxing this later, particularly if there is no type annotation.\r\n\r\n* Open question: Should we give an error for return expressions not assignable to element type (same as the question above)\r\n    * Answer: no return expressions.\r\n* If there are no yield operands and no `yield *` expressions, what should the element type be?\r\n    * Answer: implicit any\r\n\r\n## The `*` type constructor\r\n\r\nSince the Iterable type will be used a lot, it is a good opportunity to add a syntactic form for iterable types. We will use `T*` to mean `Iterable<T>`, much the same as `T[]` is `Array<T>`. It does not do anything special, it's just a shorthand. It will have the same grammatical precedence as `[]`.\r\n\r\nQuestion: Should it be an error to use `*` type if you are compiling below ES6.\r\n\r\nThe good things about this design is that it is super easy to create an iterable by declaring a generator function. And it is super easy to consume it like you would any other type of iterable.\r\n```ts\r\nfunction *g(limit) {\r\n    for (var i = 0; i < limit; i++) {\r\n        yield i;\r\n    }\r\n}\r\n\r\nfor (let i of g(100)) {\r\n    console.log(i);\r\n}\r\nvar array = [...g(50)];\r\nvar [first, second, ...rest] = g(100);\r\n```\r\n\r\n## Drawbacks of this basic design\r\n\r\n1. The type returned by a call to next is not always correct if the generator has a return expression.\r\n```ts\r\nfunction *g() {\r\n    yield 0;\r\n    return \"\";\r\n}\r\nvar instance = g();\r\nvar x = instance.next().value; // x is number, correct\r\nvar x2 = instance.next().value; // x2 is given type number, but it's actually a string!\r\n```\r\nThis implies that maybe we should give an error when return expressions are not assignable to the element type. Though if we do, there is no way out.\r\n2. The types of `yield` and `yield *` expressions are just any. Many users will not care about these, but the type of the `yield` expression is useful if for example, you are implementing await on top of yield.\r\n3. If you type your generator with the `*` type, it does not allow someone to call next directly on the generator. Instead they must cast the generator or get the iterator from the generator.\r\n```ts\r\nfunction *g(): number* {\r\n    yield 0;\r\n}\r\nvar gen = g();\r\ngen.next(); // Error, but allowed in ES6 (preferred in fact)\r\n(<IterableIterator<number>>gen).next(); // works, but really ugly\r\ngen[Symbol.iterator]().next(); // works, but pretty ugly as well\r\n```\r\nTo clarify, issue 3 is not an issue for for-of, spread, and destructuring. It is only an issue for direct calls to next. The good thing is that you can get around this by either leaving off the type annotation from the generator, or by typing it as an IterableIterator.\r\n\r\n# Advanced additions to proposal\r\nTo help alleviate issue 2, we can introduce a nominal Generator type (already in es6.d.ts today). It is an interface, but the compiler would have a special understanding of its type arguments. It would look something like this:\r\n```ts\r\ninterface Generator<TYield, TReturn, TNext> extends IterableIterator<TYield /*| TReturn*/> {\r\n    next(n: TNext): IteratorResult<TYield /*|TReturn*/>;\r\n    // throw and return methods elided\r\n}\r\n```\r\n\r\nNotice that TReturn is not used in the type, but it will have special meaning if you are using something that is nominally a Generator. Use of the Generator type annotation is purely optional. The reason that we need to omit TReturn in the next method is so that Generator can be assignable to `IterableIterator<TYield>`. Note that this means issue 1 still remains.\r\n\r\n* The type of a `yield` expression will be the type of TNext\r\n```ts\r\nfunction *g(): Generator<number, any, string> {\r\n   var x = yield 0; // x has type string\r\n}\r\n```\r\n* If the user does not specify the Generator type annotation, then consuming a yield expression **as an expression** will be an implicit any. Yield expression statements will be unaffected.\r\n\r\n* For a return expression not assignable to the yield type of the generator, we can give an error (require a type annotation) or we can infer `Generator<TYield, TReturn, any>`?\r\n```ts\r\nfunction *g() {\r\n    yield 0;\r\n    return \"\"; // Error or infer TReturn as string\r\n}\r\n```\r\n\r\nOnce we have TReturn in place, the following rules are added:\r\n* If the operand of `yield *` is a Generator, then the `yield *` expression has the type TReturn (the second type argument of that generator)\r\n* If the operand of a `yield *` is a Generator, and the `yield *` expression is inside a Generator, TNext of the outer generator must be assignable to TNext of the inner one.\r\n```ts\r\nfunction *g1(): Generator<any, any, string> {\r\n    var t = yield * g2(); // Error that string is not assignable to number\r\n}\r\nfunction *g2(): Generator<any, any, number> {\r\n    var s = yield 0;\r\n}\r\n```\r\n* If the operand of `yield *` is not a Generator, and the `yield *` is used as an expression, it will be an implicit any.\r\n\r\nOk, now for issue 1, the incorrectness of next. There is no great way to do this. But one idea, courtesy of @CyrusNajmabadi, is to use TReturn in the body of the Generator interface, so that it looks like this:\r\n```ts\r\ninterface Generator<TYield, TReturn, TNext> extends IterableIterator<TYield> {\r\n    next(n: TNext): IteratorResult<TYield | TReturn>;\r\n    // throw and return methods elided\r\n}\r\n```\r\n\r\nAs it is, Generator will not be assignable to `IterableIterator<TYield>`. To make it assignable, we would change assignability so that every time we assign `Generator<TYield, TReturn, TNext>` to something, assignability changes this to `Generator<TYield, any, TNext>` for the purposes of the assignment. This is very easy to do in the compiler.\r\n\r\nWhen we do this, we get the following result:\r\n```ts\r\nfunction *g() {\r\n    yield 0;\r\n    return \"\";\r\n}\r\nvar g1 = g();\r\nvar x1 = g1.next().value; // number | string (was number with old typing)\r\nvar x2 = g1.next().value; // number | string (was number with old typing, and should be string)\r\n\r\nvar g2: Iterator<number> = g(); // Assignment is allowed by special rule!\r\nvar x3 = g2.next(); // number, correct\r\nvar x4 = g2.next(); // number, should be string\r\n```\r\nSo you lose the correctness of next when you subsume the generator into an iterable/iterator. But you at least get general correctness when you are using it raw, as a generator.\r\n\r\nAdditionally, operators like for-of, spread, and destructuring would just get TYield, and would be unaffected by this addition, including if they are done on a Generator.\r\n\r\nThank you to everyone who helped come up with these ideas."},"comment":{"url":"https://api.github.com/repos/Microsoft/TypeScript/issues/comments/97704978","html_url":"https://github.com/Microsoft/TypeScript/issues/2873#issuecomment-97704978","issue_url":"https://api.github.com/repos/Microsoft/TypeScript/issues/2873","id":97704978,"user":{"login":"Griffork","id":3665998,"avatar_url":"https://avatars.githubusercontent.com/u/3665998?v=3","gravatar_id":"","url":"https://api.github.com/users/Griffork","html_url":"https://github.com/Griffork","followers_url":"https://api.github.com/users/Griffork/followers","following_url":"https://api.github.com/users/Griffork/following{/other_user}","gists_url":"https://api.github.com/users/Griffork/gists{/gist_id}","starred_url":"https://api.github.com/users/Griffork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Griffork/subscriptions","organizations_url":"https://api.github.com/users/Griffork/orgs","repos_url":"https://api.github.com/users/Griffork/repos","events_url":"https://api.github.com/users/Griffork/events{/privacy}","received_events_url":"https://api.github.com/users/Griffork/received_events","type":"User","site_admin":false},"created_at":"2015-04-30T08:21:42Z","updated_at":"2015-04-30T08:21:42Z","body":"@yortus it's one type to rule them all, not one type per yield. All yields should be one type (be that a programmer specified union or not). \r\nMore importantly; what is to the right of a yield should never influence the type of what comes to the left of a yield. That's assuming a relationship that doesn't exist and is easily proved wrong. \r\n\r\nWhen you say \"if no YieldMapper is given\" what constitutes given? Do users have to supply extra explicit typing? "}}