{"action":"opened","issue":{"url":"https://api.github.com/repos/calrissian/accumulo-recipes/issues/280","labels_url":"https://api.github.com/repos/calrissian/accumulo-recipes/issues/280/labels{/name}","comments_url":"https://api.github.com/repos/calrissian/accumulo-recipes/issues/280/comments","events_url":"https://api.github.com/repos/calrissian/accumulo-recipes/issues/280/events","html_url":"https://github.com/calrissian/accumulo-recipes/issues/280","id":54700200,"number":280,"title":"[QFDStores] Provide a mechanism for dumping out full nested schemas","user":{"login":"cjnolet","id":1242464,"avatar_url":"https://avatars.githubusercontent.com/u/1242464?v=3","gravatar_id":"","url":"https://api.github.com/users/cjnolet","html_url":"https://github.com/cjnolet","followers_url":"https://api.github.com/users/cjnolet/followers","following_url":"https://api.github.com/users/cjnolet/following{/other_user}","gists_url":"https://api.github.com/users/cjnolet/gists{/gist_id}","starred_url":"https://api.github.com/users/cjnolet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cjnolet/subscriptions","organizations_url":"https://api.github.com/users/cjnolet/orgs","repos_url":"https://api.github.com/users/cjnolet/repos","events_url":"https://api.github.com/users/cjnolet/events{/privacy}","received_events_url":"https://api.github.com/users/cjnolet/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/calrissian/accumulo-recipes/labels/new+feature","name":"new feature","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-01-18T15:46:42Z","updated_at":"2015-01-18T15:46:42Z","closed_at":null,"body":"The JSONTupleStore class gave us the ability to ingest documents from raw, arbitrarily nested json. The KeyValueIndex provides both cardinality information and key information based on the types of objects. What it does not provide is information about the nesting of objects which may have been flattened. \r\n\r\nThe direction I can see this going is an optional key in the index table (or maybe even a different table called storeType_schema) that would allow the schemas to be aggregated together as events are being persisted. A good example of how this is possible is in the StructType class in Spark SQL. This class basically holds a Set[StructField]. A StructField can be another StructType with its own StructField objects. This object gets easily marshalled to and from json and would be a great starting point for a separate schema table. \r\n\r\nThe nice thing about the schemas as well is that they can be merged togethe with a simple set union."}}