{"action":"created","issue":{"url":"https://api.github.com/repos/stan-dev/stan/issues/1681","labels_url":"https://api.github.com/repos/stan-dev/stan/issues/1681/labels{/name}","comments_url":"https://api.github.com/repos/stan-dev/stan/issues/1681/comments","events_url":"https://api.github.com/repos/stan-dev/stan/issues/1681/events","html_url":"https://github.com/stan-dev/stan/pull/1681","id":116243074,"number":1681,"title":"Feature/issue 1057 generalize indexing","user":{"login":"bob-carpenter","id":3383807,"avatar_url":"https://avatars.githubusercontent.com/u/3383807?v=3","gravatar_id":"","url":"https://api.github.com/users/bob-carpenter","html_url":"https://github.com/bob-carpenter","followers_url":"https://api.github.com/users/bob-carpenter/followers","following_url":"https://api.github.com/users/bob-carpenter/following{/other_user}","gists_url":"https://api.github.com/users/bob-carpenter/gists{/gist_id}","starred_url":"https://api.github.com/users/bob-carpenter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bob-carpenter/subscriptions","organizations_url":"https://api.github.com/users/bob-carpenter/orgs","repos_url":"https://api.github.com/users/bob-carpenter/repos","events_url":"https://api.github.com/users/bob-carpenter/events{/privacy}","received_events_url":"https://api.github.com/users/bob-carpenter/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-11-11T01:37:04Z","updated_at":"2015-11-11T17:27:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/stan-dev/stan/pulls/1681","html_url":"https://github.com/stan-dev/stan/pull/1681","diff_url":"https://github.com/stan-dev/stan/pull/1681.diff","patch_url":"https://github.com/stan-dev/stan/pull/1681.patch"},"body":"#### Summary:\r\n\r\nAllow multi-indexing in expressions and assignment statements.\r\n\r\n#### Intended Effect:\r\n\r\nAdds indexing functions, types, and extends language to allow multiple indexes (lower bounds, upper bounds, lower and upper bounds, array of int, all) to both expressions and statements.\r\n\r\nThe API code is in `stan/model/indexing` contains the following functionality:\r\n\r\n* `index.hpp`: structures for single and multiple indexing\r\n* `index_list.hpp`: sequence of indexes\r\n* `deep_copy.hpp`:  deep copy arbitrary Stan data types (avoids aliasing)\r\n* `rvalue_index_size.hpp`: metaprogram for size of result container given index and size of underlying container\r\n* `rvalue_at.hpp`: map index object plus result position to underlying position\r\n* `rvalue_return.hpp`: metaprogram for calculating return type of indexing\r\n* `lvalue.hpp`: assignment value to variable given an index list\r\n\r\nThe structures in index lists are mirrored in the abstract syntax tree (`stan/lang/ast.hpp`) and it also contains code for type inference.  There's also a new program to compute whether a variable shows up in an expression to detect the possibility of aliasing.\r\n\r\nThe C++ code generator changes are surprisingly straightforward and in the usual place (`stan/lang/generator.hpp`) --- the only hitch is checking if the left-hand variable occurs in the right-hand side expression, which causes the right side to be wrapped in a deep copy.  The harder part is generating neat code for user-facing and code-generation purposes --- there is separate logic for expressions printed for users and for C++.\r\n\r\nThe parser code is in `stan/lang/grammars/':\r\n\r\n* `indexes_grammar...`: grammar for sequence of single and multiple indexes\r\n* `statement_grammar...`: adds multiple assignment case with semantic actions\r\n* `term_grammar...`: adds case for multiple indexed expression with semantic actions\r\n\r\nIn the statement and term grammar, the original logic remains, but now no loger has expectation operators `>` but just concatenation `>>` so that failure to match can backtrack to the multiple indexing.  This means multiple indexing has to deal with informative error messages.\r\n\r\n#### How to Verify:\r\n\r\nThere are full unit tests for the boundary conditions in the API and template metaprograms.  There are also exhaustive unit tests for up to three dimensions of indexes both at the API level and the grammar/parser level that verify values and sizes are all correctly assigned.  (There's a repeated tested idiom with helper functions for expressions and statements and a whole lot of cut and paste and emacs macros in the grammar examples.)\r\n\r\n#### Side Effects:\r\n\r\nError messages are slightly different for indexing errors because they are handled in the multi-indexing blocks.  Tests have been updated to reflect this change.\r\n\r\nExpressions and assignment statements with no multiple indexing (that is, no indexing or just single integer indexes) will be parsed and code-generated as before.\r\n\r\n#### Documentation:\r\n\r\nNew manual chapter for user's guide and new chapter for reference manual;  updated BNF description and added some cross-references.\r\n\r\nAPI functions all documented.\r\n\r\n#### Reviewer Suggestions:\r\n\r\nDaniel (he'll be pickiest and this one should be looked at pretty closely)"},"comment":{"url":"https://api.github.com/repos/stan-dev/stan/issues/comments/155853951","html_url":"https://github.com/stan-dev/stan/pull/1681#issuecomment-155853951","issue_url":"https://api.github.com/repos/stan-dev/stan/issues/1681","id":155853951,"user":{"login":"syclik","id":425751,"avatar_url":"https://avatars.githubusercontent.com/u/425751?v=3","gravatar_id":"","url":"https://api.github.com/users/syclik","html_url":"https://github.com/syclik","followers_url":"https://api.github.com/users/syclik/followers","following_url":"https://api.github.com/users/syclik/following{/other_user}","gists_url":"https://api.github.com/users/syclik/gists{/gist_id}","starred_url":"https://api.github.com/users/syclik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/syclik/subscriptions","organizations_url":"https://api.github.com/users/syclik/orgs","repos_url":"https://api.github.com/users/syclik/repos","events_url":"https://api.github.com/users/syclik/events{/privacy}","received_events_url":"https://api.github.com/users/syclik/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T17:27:41Z","updated_at":"2015-11-11T17:27:41Z","body":"Yes, please. \n\n> On Nov 11, 2015, at 12:14 PM, Bob Carpenter <notifications@github.com> wrote:\n> \n> Cool, \n> \n> > ./runTests.py -j4 src/test\n> \n> seems to kick everything off. May I update the make doc while\n> I'm at it to make this more visible?\n> \n> - Bob\n> \n> ./runtests.py\n> > On Nov 10, 2015, at 11:26 PM, Daniel Lee <notifications@github.com> wrote:\n> > \n> > It's just:\n> > `./runTests.py src/test/integration`\n> > (or `./runTests.py src/test/integration/compile_models_test.cpp`)\n> > \n> > I did some make magic to get all the good models to be run through stanc\n> > then compiled with the C++ compiler.\n> > \n> > \n> > On Tue, Nov 10, 2015 at 11:19 PM, Bob Carpenter <notifications@github.com>\n> > wrote:\n> > \n> > > Nope, broken here, too. I'll fix and push a patch.\n> > >\n> > > Is there a make target for the integration tests? I don't see\n> > > it with \"make help\" --- shouldn't it be under \"Tests:\"?\n> > >\n> > > - Bob\n> > >\n> > > > On Nov 10, 2015, at 10:47 PM, Daniel Lee <notifications@github.com>\n> > > wrote:\n> > > >\n> > > > I take it back... Bob, does this run on your machine:\n> > > > make src/test/test-models/good/indexing/good-ints.hpp-test\n> > > >\n> > > > It first compiles src/test/test-models/good/indexing/good-ints.stan to\n> > > src/test/test-models/good/indexing/good-ints.hpp. Then it tries to compile\n> > > the header file.\n> > > >\n> > > > Mine doesn't compile.\n> > >\n> > > >\n> > > > —\n> > > > Reply to this email directly or view it on GitHub.\n> > > >\n> > >\n> > > —\n> > > Reply to this email directly or view it on GitHub\n> > > <https://github.com/stan-dev/stan/pull/1681#issuecomment-155657016>.\n> > >\n> > —\n> > Reply to this email directly or view it on GitHub.\n> > \n> \n> —\n> Reply to this email directly or view it on GitHub.\n> \n"}}