{"action":"created","issue":{"url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/273","labels_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/273/labels{/name}","comments_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/273/comments","events_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/273/events","html_url":"https://github.com/CausalityLtd/ponyc/issues/273","id":98498351,"number":273,"title":"Syntax sugar for lambda with immediate partial application of arguments. ","user":{"login":"jemc","id":4623870,"avatar_url":"https://avatars.githubusercontent.com/u/4623870?v=3","gravatar_id":"","url":"https://api.github.com/users/jemc","html_url":"https://github.com/jemc","followers_url":"https://api.github.com/users/jemc/followers","following_url":"https://api.github.com/users/jemc/following{/other_user}","gists_url":"https://api.github.com/users/jemc/gists{/gist_id}","starred_url":"https://api.github.com/users/jemc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jemc/subscriptions","organizations_url":"https://api.github.com/users/jemc/orgs","repos_url":"https://api.github.com/users/jemc/repos","events_url":"https://api.github.com/users/jemc/events{/privacy}","received_events_url":"https://api.github.com/users/jemc/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/CausalityLtd/ponyc/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"closed","locked":false,"assignee":{"login":"andymcn","id":7050628,"avatar_url":"https://avatars.githubusercontent.com/u/7050628?v=3","gravatar_id":"","url":"https://api.github.com/users/andymcn","html_url":"https://github.com/andymcn","followers_url":"https://api.github.com/users/andymcn/followers","following_url":"https://api.github.com/users/andymcn/following{/other_user}","gists_url":"https://api.github.com/users/andymcn/gists{/gist_id}","starred_url":"https://api.github.com/users/andymcn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andymcn/subscriptions","organizations_url":"https://api.github.com/users/andymcn/orgs","repos_url":"https://api.github.com/users/andymcn/repos","events_url":"https://api.github.com/users/andymcn/events{/privacy}","received_events_url":"https://api.github.com/users/andymcn/received_events","type":"User","site_admin":false},"milestone":null,"comments":10,"created_at":"2015-08-01T01:57:48Z","updated_at":"2015-09-09T05:59:56Z","closed_at":"2015-09-08T20:34:32Z","body":"Many other languages and frameworks use anonymous function literals or lambdas to specify a function to be applied when an asynchronous operation completes.  In [the recent implementation of \"promises\" in Pony](https://github.com/CausalityLtd/ponyc/tree/master/packages/promises), a similar approach is used to supply a function to be applied when a promise is fulfilled or rejected.\r\n\r\nIn practice, it is usually useful to send some bit of state with the function in order to facilitate an appropriate response action.  Some languages allow lambdas to \"close\" on variables in their surrounding lexical scope and essentially carry around a reference to the stack frame, so that local variables in that scope can be used in the response.  Pony lambdas do not do this, but a similar effect can be achieved with partial application of arguments to a function.  This is particularly useful when all applied arguments are _sendable_, as the function is most often sent to another actor for asynchronous execution.  The `term/readline` package, which uses `promises`, demonstrates [an example of this](https://github.com/CausalityLtd/ponyc/blob/5c51732322acca75d1455e728db85c161efb416c/packages/term/readline.pony#L303-L306), though it does so by using out-of-line defined `tag` functions instead of in-line lambda literals.\r\n\r\nHowever, there are many reasons why using in-line lambda literals may be preferable in certain cases, including visual localization of logic and a tidier namespace of defined functions in the surrounding class/actor/primitive.  This is also possible in Pony, and works quite well for one-off \"reaction\" functions as shown in the following [real-world example from a test suite for a package I'm working on](https://github.com/jemc/pony-zmq/blob/18e6fcbff5360da25789b19d58e336c416c6626f/zmq/test/socket_test.pony):\r\n\r\n```pony\r\nclass SocketTest is UnitTest\r\n  let _env: Env\r\n  new iso create(env: Env) => _env = env\r\n  fun name(): String => \"zmq.Socket\"\r\n  \r\n  fun apply(h: TestHelper): TestResult =>\r\n    let ra = _SocketReactor; let a = zmq.Socket(zmq.PAIR, ra.notify())\r\n    let rb = _SocketReactor; let b = zmq.Socket(zmq.PAIR, rb.notify())\r\n    \r\n    a.bind(\"tcp://localhost:8899\")\r\n    b.connect(\"tcp://localhost:8899\")\r\n    a.send_string(\"foo\")\r\n    b.send_string(\"bar\")\r\n    \r\n    ra.next(recover lambda(h: TestHelper, a: zmq.Socket, m: zmq.Message) =>\r\n      h.expect_eq[zmq.Message](m, recover zmq.Message.push(\"foo\") end)\r\n      a.dispose()\r\n    end~apply(h,a) end)\r\n    \r\n    rb.next(recover lambda(h: TestHelper, b: zmq.Socket, m: zmq.Message) =>\r\n      h.expect_eq[zmq.Message](m, recover zmq.Message.push(\"bar\") end)\r\n      b.dispose()\r\n    end~apply(h,b) end)\r\n    \r\n    ra.when_closed(recover lambda(h: TestHelper, rb: _SocketReactor) =>\r\n      rb.when_closed(recover lambda(h: TestHelper) =>\r\n        h.complete(true)\r\n      end~apply(h) end)\r\n    end~apply(h,rb) end)\r\n    \r\n    LongTest\r\n```\r\n\r\nAs mentioned above, this works well, but has a few aspects of its syntax that are less-than-ideal:\r\n- The variables for immediate partial application have to be specified both in the arguments to `~apply` and in the parameters of the `lambda` literal.  For long lambdas, these two locations become even more delocalized from one another.\r\n- The types of the variables for immediate partial application are already known to the compiler from the surrounding scope, but the syntax requires them to be written again in the parameter signature of the `lambda` literal.\r\n- The intent is to create a `lambda` of a specific \"arity\" that _also_ has access to specific captures, but this intent is obscured and not obvious because one must mentally \"subtract\" the partial application arity from the parameter arity.  For example, in each of the calls to `next` in the example snippet, the intent is to create and pass a lambda that accepts a single `zmq.Message` parameter and _also_ has access to the `h` and `a` variables as immediate captures, but this intent is obscured by the three-parameter signature.  To put it another way, the parameter list contains two very different kinds of entities - parameters and immediate captures - and this distinction is not obvious without also studying the partial application at the end of the lambda literal.\r\n\r\nI would argue that as the Pony user base and available libraries expand, this usage pattern has the potential to become quite useful and popular, but without additional syntax sugar it is more cumbersome than it needs to be.  In addition to syntactical benefits, the implementation of a lambda created with immediate captures has the potential to be more execution-efficient than the alternative of having a lambda literal, then applying some partial arguments as a second step (this is only speculation, as I'm not intimately familiar with the implementation of the current `lambda` sugar yet).\r\n\r\nTo that end, I propose that syntax sugar be added that allows for immediate partial application of local variables to be specified at the _head_ of the `lambda` literal, and without having to re-specify them (or their types) in the parameter list.  Within the body of `lambda` literal, the named variables would be available with the same names and types that they had in the body of the enclosing function.  Other than those specifications, I'm not fixated on the particular details of the syntax, though I've prepared an example to get the conversation started, and I'm interested to hear other ideas and suggestions.  Here is the above example, rewritten with one conception of how this syntax sugar could look:\r\n\r\n```pony\r\nclass SocketTest is UnitTest\r\n  let _env: Env\r\n  new iso create(env: Env) => _env = env\r\n  fun name(): String => \"zmq.Socket\"\r\n  \r\n  fun apply(h: TestHelper): TestResult =>\r\n    let ra = _SocketReactor; let a = zmq.Socket(zmq.PAIR, ra.notify())\r\n    let rb = _SocketReactor; let b = zmq.Socket(zmq.PAIR, rb.notify())\r\n    \r\n    a.bind(\"tcp://localhost:8899\")\r\n    b.connect(\"tcp://localhost:8899\")\r\n    a.send_string(\"foo\")\r\n    b.send_string(\"bar\")\r\n    \r\n    ra.next(recover lambda~(h,a)(m: zmq.Message) =>\r\n      h.expect_eq[zmq.Message](m, recover zmq.Message.push(\"foo\") end)\r\n      a.dispose()\r\n    end end)\r\n    \r\n    rb.next(recover lambda~(h,b)(m: zmq.Message) =>\r\n      h.expect_eq[zmq.Message](m, recover zmq.Message.push(\"bar\") end)\r\n      b.dispose()\r\n    end end)\r\n    \r\n    ra.when_closed(recover lambda~(h,rb)() =>\r\n      rb.when_closed(recover lambda~(h)() =>\r\n        h.complete(true)\r\n      end end)\r\n    end end)\r\n    \r\n    LongTest\r\n```\r\n\r\nSpecifically, this proposal would amend lambda syntax sugar to accept an optional set of immediate captures preceding the parameter signature, marked by the `~` character to denote its distinction from the parameter signature syntax.  To me, it makes conceptual sense that the immediate captures precede the parameter signature because in the partial application method they become the parameters that precede the \"remaining\" parameters.\r\n\r\nAs mentioned above, this is only one idea for how this might look, and I'm interested to hear other ideas for the details of the syntax and feedback from the maintainers about whether this is find of syntax desirable as proposed.  I hope you'll agree that it's helpful; I suspect such a feature would have a significant impact on time and effort required to create and maintain a robust, comprehensive set of tests for this socket library.  Also, I'm willing to put forth the development effort to implement the sugar myself if that makes a difference - I do have some experience working with other parsers, ASTs and compilers."},"comment":{"url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/comments/138794417","html_url":"https://github.com/CausalityLtd/ponyc/issues/273#issuecomment-138794417","issue_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/273","id":138794417,"user":{"login":"jemc","id":4623870,"avatar_url":"https://avatars.githubusercontent.com/u/4623870?v=3","gravatar_id":"","url":"https://api.github.com/users/jemc","html_url":"https://github.com/jemc","followers_url":"https://api.github.com/users/jemc/followers","following_url":"https://api.github.com/users/jemc/following{/other_user}","gists_url":"https://api.github.com/users/jemc/gists{/gist_id}","starred_url":"https://api.github.com/users/jemc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jemc/subscriptions","organizations_url":"https://api.github.com/users/jemc/orgs","repos_url":"https://api.github.com/users/jemc/repos","events_url":"https://api.github.com/users/jemc/events{/privacy}","received_events_url":"https://api.github.com/users/jemc/received_events","type":"User","site_admin":false},"created_at":"2015-09-09T05:59:56Z","updated_at":"2015-09-09T05:59:56Z","body":"So far, this is working great, aside from one snag.  In my original example code, there is one part that doesn't transition fully smoothly.  That is, I get a compiler error for something I think should be possible to do safely.  Here is the original code (where `ra`, `rb`, and `h`, are all local tag references to actors):\r\n```pony\r\n    ra.when_closed(lambda iso(h: TestHelper, rb: _SocketReactor)(h,rb) =>\r\n      rb.when_closed(lambda iso(h: TestHelper)(h) =>\r\n        h.complete(true)\r\n      end~apply(h) end)\r\n    end~apply(h,rb) end)\r\n```\r\n\r\nWith the new syntax, here is how I expect to be able to rewrite the code:\r\n```pony\r\n    ra.when_closed(lambda iso()(h,rb) =>\r\n      rb.when_closed(lambda iso()(h) =>\r\n        h.complete(true)\r\n      end)\r\n    end)\r\n```\r\n\r\nBut I get the following compiler error:\r\n```pony\r\n/home/jemc/1/code/hg/pony-zmq/zmq/test/socket_transport_tests.pony:67:35: cannot capture \"h\", can only capture fields, parameters and local variables\r\n      rb.when_closed(lambda iso()(h) =>\r\n```\r\n\r\nObviously there is an issue here with directly nesting captures.  It seems like this should work, as `h` here is a `let` field in the final formulation of the outer lambda's object literal representation, and it is a `tag`, so it should be safe to alias as such.  My guess is that the outer lambda's object literal representation is not full resolved or \"in place\" at the time when the compiler tries to find the definition of `h` for the inner capture.\r\n\r\nNote that the following example works, where I use the `h' = h` form of capture (but is unnecessarily cumbersome unless there is a technical reason why the above example cannot be made to work as shown):\r\n```pony\r\n    ra.when_closed(lambda iso()(h,rb) =>\r\n      rb.when_closed(lambda iso()(h' = h) =>\r\n        h'.complete(true)\r\n      end)\r\n    end)\r\n```\r\n"}}