{"action":"created","issue":{"url":"https://api.github.com/repos/json-api/json-api/issues/940","labels_url":"https://api.github.com/repos/json-api/json-api/issues/940/labels{/name}","comments_url":"https://api.github.com/repos/json-api/json-api/issues/940/comments","events_url":"https://api.github.com/repos/json-api/json-api/issues/940/events","html_url":"https://github.com/json-api/json-api/issues/940","id":120949339,"number":940,"title":"Proposal for recursive includes","user":{"login":"ziege","id":1983424,"avatar_url":"https://avatars.githubusercontent.com/u/1983424?v=3","gravatar_id":"","url":"https://api.github.com/users/ziege","html_url":"https://github.com/ziege","followers_url":"https://api.github.com/users/ziege/followers","following_url":"https://api.github.com/users/ziege/following{/other_user}","gists_url":"https://api.github.com/users/ziege/gists{/gist_id}","starred_url":"https://api.github.com/users/ziege/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ziege/subscriptions","organizations_url":"https://api.github.com/users/ziege/orgs","repos_url":"https://api.github.com/users/ziege/repos","events_url":"https://api.github.com/users/ziege/events{/privacy}","received_events_url":"https://api.github.com/users/ziege/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-12-08T07:16:53Z","updated_at":"2015-12-22T04:53:00Z","closed_at":null,"body":"## Recursive Includes\r\nThe inclusion of related resources currently doesn't support the easy inclusion of recursive resource data, stored in a tree-like structure. This is a known issue (see #215) without any solution yet.\r\n\r\n### What's possible today\r\nIt's possible to explicitly add parent/child resources to the include parameter, but this always require to know of the exact number of available parent/child levels.\r\n\r\nSo you can e.g. request the data for the location of an event, including data about the five parent location levels:\r\n```\r\nGET /events/1?include=location.parent.parent.parent.parent.parent HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\n### What's missing\r\nThe current solution doesn't offer a way to get the data of all parents without knowing the number of parent levels. And the value of the include parameter can get very long.\r\n\r\nWe need a way to easily express how many parent/child levels should be fetched, including a way to define an infinite number of levels.\r\n\r\n### Suggested syntax\r\nSquare brackets `[` (U+005B LEFT SQUARE BRACKET) and `]` (U+005D RIGHT SQUARE BRACKET) are added to the relationship path, containing a positive number that defines how often the last part ahead the brackets should be fetched in recursion.\r\n\r\nSo the previous example would look like this;\r\n```\r\nGET /events/1?include=location.parent[5] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\nIf you want all levels but don't know the exact number of available levels, you can use an asterisk `*` (U+002A ASTERISK) instead of a number:\r\n```\r\nGET /events/1?include=location.parent[*] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\nThe server includes the `location` (implicitly) and its `parent` (location). Then it repeats the fetching of `parent` locations (N-1)/infinite times for all resulting resources from the given relationship path, as long as `parent` is set.\r\n\r\nNote: Leaving the brackets means the number of recursion levels is set to 1, so the behavior is backwards compatible.\r\n\r\n#### Combining recursive includes\r\nTo allow the inclusion of resources that depend on recursive includes, also the following relationship path is allowed to e.g. get the `category` resource for the three levels of location parents:\r\n```\r\nGET /events/1?include=location.parent[3].category HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\nAnd if this category also allows recursion, this can be used too:\r\n```\r\nGET /events/1?include=location.parent[3].category.parent[*] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\n#### Using multiple recursive includes\r\nOf course you can also do multiple recursive includes, e.g. if you want to fetch the parents and children together with a category:\r\n```\r\nGET /category/1?include=parent[*],children[2] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\n#### Indirect relations / group repetition\r\nBy default the aforementioned syntax repeats the part of the relation path ahead the counter in the square brackets - but in some cases this isn't enough, because multiple parts of the relation path need to be fetched recursively.\r\n\r\nFor example a user resource could have connections to friends and these friend resources would refer back to user resources, who again refer to friend resources and so on:\r\n\r\n```json\r\n{\r\n  \"data\": {\r\n    \"id\": \"301\",\r\n    \"type\": \"user\",\r\n    \"relationships\": {\r\n      \"friends\": {\r\n        \"data\": [\r\n          {\"id\": \"200\", \"type\": \"friend\"}\r\n        ]\r\n      }\r\n    }\r\n  },\r\n  \"included\": [\r\n    {\r\n      \"id\": \"200\",\r\n      \"type\": \"friend\",\r\n      \"attributes\": {\r\n          \"added\": \"2015-12-01 13:34:00Z\"\r\n      },\r\n      \"relationships\": {\r\n        \"user\": {\r\n          \"data\": {\"id\": \"302\", \"type\": \"user\"}\r\n        }\r\n      }\r\n    },\r\n    {\r\n      \"id\": \"302\",\r\n      \"type\": \"user\",\r\n      \"relationships\": {\r\n        \"friends\": {\r\n          \"data\": [\r\n            {\"id\": \"201\", \"type\": \"friend\"},\r\n            {\"id\": \"202\", \"type\": \"friend\"}\r\n          ]\r\n        }\r\n      }\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\nTo fetch multiple levels of friend and users resources, the relations to repeat are wrapped by additional square brackets `[` (U+005B LEFT SQUARE BRACKET) and `]` (U+005D RIGHT SQUARE BRACKET).\r\n\r\n```\r\nGET /users/301?include=[friends.user][3] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\nThe server include the `friends` (implicitly) and the related `user`. Then it repeats the fetching of `friends` and their related `user` resources (N-1)/infinite times for all resulting `user` resources from the given relationship path, as long as `friends` and `user` are set.\r\n\r\nFor the location example, the request could also be written like this, but the grouping has no effect here:\r\n```\r\nGET /events/1?include=location.[parent][5] HTTP/1.1\r\nAccept: application/vnd.api+json\r\n```\r\n\r\n### Handling of special cases\r\n\r\n#### Limitations on server side\r\nIn the previous example, a `user` can have multiple `friends`, which means the number of included resources increases exponentially with each recursion level - so the resulting document could get really huge (more than 100,000 resources after 2-3 levels possible in this case).\r\n\r\nTo avoid this, the server __SHOULD__ limit the number of allowed inclusion levels in such cases (on a per type basis) and reply with the status code `400 Bad Request`, including error details to describe the reason for the error and the defined limits.\r\n\r\nThis can also be used by the server to disallow recursive includes globally.\r\n\r\n#### Repeating resources/Infinite recursion\r\nRecursive includes can lead to resources that are linked multiple times. Also an infinite recursion could be possible. The server __MUST__ check if an resource with the same `type` and `id` has already been included before and include and process it only once.\r\n\r\nClients __SHOULD__ also check for infinite recursion to avoid problems. It's up to the client how to handle such data."},"comment":{"url":"https://api.github.com/repos/json-api/json-api/issues/comments/166504652","html_url":"https://github.com/json-api/json-api/issues/940#issuecomment-166504652","issue_url":"https://api.github.com/repos/json-api/json-api/issues/940","id":166504652,"user":{"login":"toobulkeh","id":1342064,"avatar_url":"https://avatars.githubusercontent.com/u/1342064?v=3","gravatar_id":"","url":"https://api.github.com/users/toobulkeh","html_url":"https://github.com/toobulkeh","followers_url":"https://api.github.com/users/toobulkeh/followers","following_url":"https://api.github.com/users/toobulkeh/following{/other_user}","gists_url":"https://api.github.com/users/toobulkeh/gists{/gist_id}","starred_url":"https://api.github.com/users/toobulkeh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/toobulkeh/subscriptions","organizations_url":"https://api.github.com/users/toobulkeh/orgs","repos_url":"https://api.github.com/users/toobulkeh/repos","events_url":"https://api.github.com/users/toobulkeh/events{/privacy}","received_events_url":"https://api.github.com/users/toobulkeh/received_events","type":"User","site_admin":false},"created_at":"2015-12-22T04:53:00Z","updated_at":"2015-12-22T04:53:00Z","body":"My 2 cents from the Rails-world (read: RDBS, not NoSQL) on unlimited recursion design:\r\n\r\n```ruby\r\nresources :articles do\r\n  resources :comments, only: [:index, :new, :create]\r\nend\r\nresources :comments, only: [:show, :edit, :update, :destroy]\r\n```\r\n\r\n> One way to avoid deep nesting (as recommended above) is to generate the collection actions scoped under the parent, so as to get a sense of the hierarchy, but to not nest the member actions. In other words, to only build routes with the minimal amount of information to uniquely identify the resource\r\n[RailsGuides](http://guides.rubyonrails.org/routing.html#nested-resources) scroll to section 2.7.2\r\n\r\nIn practice, I think this makes a lot of sense. It's one of the driving pieces from GitHub's API spec I find comforting. Though they do break this as a steadfast rule (Organizations/Repos/...) but that's more for data-scoping-at-large.\r\n\r\nNote: Document-driven databases (NoSQL) are a whole different ballgame, as they're optimized quite differently on data relationships.\r\n\r\nIn summary, flexibility is good, but specifications are to tighten flexibility, and this is where the end of my current level of understanding ends. I hope this helps."}}