{"action":"created","issue":{"url":"https://api.github.com/repos/ekg/vg/issues/112","labels_url":"https://api.github.com/repos/ekg/vg/issues/112/labels{/name}","comments_url":"https://api.github.com/repos/ekg/vg/issues/112/comments","events_url":"https://api.github.com/repos/ekg/vg/issues/112/events","html_url":"https://github.com/ekg/vg/issues/112","id":109272703,"number":112,"title":"VG RDF generate stablish identifiers for nodes and paths","user":{"login":"JervenBolleman","id":986133,"avatar_url":"https://avatars.githubusercontent.com/u/986133?v=3","gravatar_id":"","url":"https://api.github.com/users/JervenBolleman","html_url":"https://github.com/JervenBolleman","followers_url":"https://api.github.com/users/JervenBolleman/followers","following_url":"https://api.github.com/users/JervenBolleman/following{/other_user}","gists_url":"https://api.github.com/users/JervenBolleman/gists{/gist_id}","starred_url":"https://api.github.com/users/JervenBolleman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JervenBolleman/subscriptions","organizations_url":"https://api.github.com/users/JervenBolleman/orgs","repos_url":"https://api.github.com/users/JervenBolleman/repos","events_url":"https://api.github.com/users/JervenBolleman/events{/privacy}","received_events_url":"https://api.github.com/users/JervenBolleman/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-10-01T10:29:00Z","updated_at":"2015-10-24T20:15:32Z","closed_at":null,"body":"Currently the IRI identifying a node in the RDF representation of a VG is determined by order of serialization. This means that at this time the IRIs are not global identifiers that one can use to place a VG rdf document on the web.\r\n\r\nA better IRI generation scheme would perhaps use a hash of the input so that identical input generates identical IRIs.\r\n\r\nUUIDs could be used for the IRIs but they compress very badly, they are hard to regenerate in an identical version."},"comment":{"url":"https://api.github.com/repos/ekg/vg/issues/comments/150847326","html_url":"https://github.com/ekg/vg/issues/112#issuecomment-150847326","issue_url":"https://api.github.com/repos/ekg/vg/issues/112","id":150847326,"user":{"login":"JervenBolleman","id":986133,"avatar_url":"https://avatars.githubusercontent.com/u/986133?v=3","gravatar_id":"","url":"https://api.github.com/users/JervenBolleman","html_url":"https://github.com/JervenBolleman","followers_url":"https://api.github.com/users/JervenBolleman/followers","following_url":"https://api.github.com/users/JervenBolleman/following{/other_user}","gists_url":"https://api.github.com/users/JervenBolleman/gists{/gist_id}","starred_url":"https://api.github.com/users/JervenBolleman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JervenBolleman/subscriptions","organizations_url":"https://api.github.com/users/JervenBolleman/orgs","repos_url":"https://api.github.com/users/JervenBolleman/repos","events_url":"https://api.github.com/users/JervenBolleman/events{/privacy}","received_events_url":"https://api.github.com/users/JervenBolleman/received_events","type":"User","site_admin":false},"created_at":"2015-10-24T20:15:32Z","updated_at":"2015-10-24T20:15:32Z","body":"I am not a fan of the idea of ga4gh/schemas#444 I will write something there in the next week or so.\r\n\r\nFor now a small braindump.\r\n\r\nThe aim is that the same input gives the same ids. In  RDF like XML ids are namespaced, e.g. <ftp://example.org/GRCh38/VG/byjerven/2015/Chr1/node/XXXXXX> and <ftp://example.org/GRCh38/VG/byekg/2015/Chr2/node/XXXXXX>. The namespaces ensure (normally) that global id collisions don't occur. These namespaces are long but are 100% compressible in any sane file format and storage system. \r\n\r\nWe do need a way to number the nodes in the graph in a predictable way. This means we a stable graph algorithm that visits each node at least once in a repeatable manner. Then we just number the nodes from 1 to ∞.\r\n\r\nThen every edit we make we just increment the id sequence by 1 for all new nodes. Then given the history of edits we can always rebuild the same node graph. The fact that one node is split into multiple nodes does not mean that the old node needs to be deleted, for many purposes it can just be disconnected from the global graph. Maybe with new predicates, was attachedOn5PrimeEndOf. These disconnected nodes can be taken out of RAM and logged to disk.\r\n\r\nAn other option is if the original node was named <ftp://example.org/GRCh38/VG/byekg/2015/Chr2/node/12345> and it is split into two we can call the new nodes for the split point. e.g. assume node ```ex:12345``` has sequence ```actgactg``` and is split into two for example ```act``` and ```gactg``` then we name those nodes ```ex:12345s0l3``` and ```ex:12345s3l5```. In other words once we have our first graph we have a tree encoding for determining new node ids. The nice things is that in a sorted list these kinds of ids compress really nicely.\r\n\r\nOf course these are only external node ids, in memory they are just a integers/pointers to a lookup table. Which can be stored very efficiently using delta encodings for most nodes.\r\n\r\nThe nice thing is that most node edits are data driven, e.g. by adding a new sample file you need new nodes for the side graph to work. Then while all the nodes in the original graph as well as their split childeren are in the original namespace, all the new nodes can put into a new namespace. \r\ne.g. something like <https://example.org/uk1000/patient1/Chr1/node/XXXXXX> :after <ftp://example.org/official/GRCh38/Chr1/node/YYYYYY>.\r\n\r\nIt definitely allows us to merge different VG from different sources, and we can deal with weird cases such as cancer genomes, HELA lines etc... being merged into a super graph.\r\n\r\n\r\n \r\n"}}