{"action":"created","issue":{"url":"https://api.github.com/repos/ensime/ensime-server/issues/1133","labels_url":"https://api.github.com/repos/ensime/ensime-server/issues/1133/labels{/name}","comments_url":"https://api.github.com/repos/ensime/ensime-server/issues/1133/comments","events_url":"https://api.github.com/repos/ensime/ensime-server/issues/1133/events","html_url":"https://github.com/ensime/ensime-server/issues/1133","id":114376602,"number":1133,"title":"Graph Indexer","user":{"login":"fommil","id":1914041,"avatar_url":"https://avatars.githubusercontent.com/u/1914041?v=3","gravatar_id":"","url":"https://api.github.com/users/fommil","html_url":"https://github.com/fommil","followers_url":"https://api.github.com/users/fommil/followers","following_url":"https://api.github.com/users/fommil/following{/other_user}","gists_url":"https://api.github.com/users/fommil/gists{/gist_id}","starred_url":"https://api.github.com/users/fommil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fommil/subscriptions","organizations_url":"https://api.github.com/users/fommil/orgs","repos_url":"https://api.github.com/users/fommil/repos","events_url":"https://api.github.com/users/fommil/events{/privacy}","received_events_url":"https://api.github.com/users/fommil/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ensime/ensime-server/labels/OMG!%20Ponies!","name":"OMG! Ponies!","color":"F433FF"}],"state":"open","locked":false,"assignee":{"login":"fommil","id":1914041,"avatar_url":"https://avatars.githubusercontent.com/u/1914041?v=3","gravatar_id":"","url":"https://api.github.com/users/fommil","html_url":"https://github.com/fommil","followers_url":"https://api.github.com/users/fommil/followers","following_url":"https://api.github.com/users/fommil/following{/other_user}","gists_url":"https://api.github.com/users/fommil/gists{/gist_id}","starred_url":"https://api.github.com/users/fommil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fommil/subscriptions","organizations_url":"https://api.github.com/users/fommil/orgs","repos_url":"https://api.github.com/users/fommil/repos","events_url":"https://api.github.com/users/fommil/events{/privacy}","received_events_url":"https://api.github.com/users/fommil/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/ensime/ensime-server/milestones/8","html_url":"https://github.com/ensime/ensime-server/milestones/Stable%201.0","labels_url":"https://api.github.com/repos/ensime/ensime-server/milestones/8/labels","id":916638,"number":8,"title":"Stable 1.0","description":"A stable release which will have permanent versions of the sbt-plugin and server for scala 2.9.2, 2.9.3, 2.10 and 2.11 in maven central, and a corresponding stable release in MELPA.\r\n\r\nThis will be the last version to support scala 2.9 and only serious bugfixes will cause new versions to be released.\r\n\r\nWhen released, the rolling release will become more experimental and the protocol is likely to be redesigned.\r\n\r\nTickets should only be added to this milestone if you actively plan to work on them.","creator":{"login":"fommil","id":1914041,"avatar_url":"https://avatars.githubusercontent.com/u/1914041?v=3","gravatar_id":"","url":"https://api.github.com/users/fommil","html_url":"https://github.com/fommil","followers_url":"https://api.github.com/users/fommil/followers","following_url":"https://api.github.com/users/fommil/following{/other_user}","gists_url":"https://api.github.com/users/fommil/gists{/gist_id}","starred_url":"https://api.github.com/users/fommil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fommil/subscriptions","organizations_url":"https://api.github.com/users/fommil/orgs","repos_url":"https://api.github.com/users/fommil/repos","events_url":"https://api.github.com/users/fommil/events{/privacy}","received_events_url":"https://api.github.com/users/fommil/received_events","type":"User","site_admin":false},"open_issues":15,"closed_issues":62,"state":"open","created_at":"2014-12-29T12:44:02Z","updated_at":"2015-10-31T00:23:23Z","due_on":"2015-03-30T23:00:00Z","closed_at":null},"comments":34,"created_at":"2015-10-31T00:23:23Z","updated_at":"2015-11-01T10:09:13Z","closed_at":null,"body":"Many features are not possible without additional information being made available by the Indexer, e.g. #1128, #425, #339, #1131, #1132, #655, #355\r\n\r\nIt is becoming clear that although Lucene is an excellent way to provide the interface for humans into the data that we store (and has use beyond our current implementation, e.g. #472), a relational database is simply not the right technology to store the data that we have (sorry @cvogt ! I know you did some great work last week with the Slick upgrade).\r\n\r\nTherefore, I propose that we swap out the SQL backend for a Graph Database, as the data we have really follows this form:\r\n\r\n![img_20151030_235139](https://cloud.githubusercontent.com/assets/1914041/10860529/97b97d32-7f61-11e5-9a97-65c1db88a3df.jpg)\r\n\r\nWe have four categories of Things that we want to store:\r\n\r\n- Files (either classfiles or jar/zip archives)\r\n  - `path: File` canonical on disk\r\n\r\n- Classes (flat subset of our current `RawClassfile`, includes interfaces and traits)\r\n  - `source: URI` canonical on disk\r\n  - `fqn: FullyQualifiedName`\r\n  - `access: Access`\r\n\r\n- Fields (flat subset of our current `RawField`)\r\n  - `fqn: FullyQualifiedName`\r\n  - `access: Access`\r\n\r\n- Methods (flat subset of our current `RawMethod`)\r\n  - `fqn: FullyQualifiedName`\r\n  - `line: Int`\r\n  - `descriptor: Descriptor` (not sure this is needed)\r\n  - `access: Access`\r\n\r\nwith the following relationship layers between the above:\r\n\r\n- Classes has references to the binary file that they are from (layer 1).\r\n- Methods have references to their owning class (layer 2), as well as the classes, fields and methods that are part of their signature and body (layer 3).\r\n- Fields have references to their owning class (layer 4), as well as their class membership (layer 5).\r\n- Classes have references to to their direct and interface parents (layer 6).\r\n\r\nThis simple JVM representation could be extended to include annotations, generic information and output from the scala pickler, but its a good starting point.\r\n\r\nSuch a structure allows us to ask all the questions we can currently ask plus:\r\n\r\n- \"what is the class hierarchy of X\"\r\n- \"what implements interface Y?\"\r\n- \"what uses field / method Z\"\r\n\r\nalong with a bunch of other clever stuff we could do such as path finding and principal components analysis.\r\n\r\n(Completing this ticket would also be a good opportunity to close off #1117)\r\n\r\n<bountysource-plugin>\r\n---\r\nWant to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/27847865-graph-indexer?utm_campaign=plugin&utm_content=tracker%2F239449&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F239449&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>"},"comment":{"url":"https://api.github.com/repos/ensime/ensime-server/issues/comments/152814266","html_url":"https://github.com/ensime/ensime-server/issues/1133#issuecomment-152814266","issue_url":"https://api.github.com/repos/ensime/ensime-server/issues/1133","id":152814266,"user":{"login":"fommil","id":1914041,"avatar_url":"https://avatars.githubusercontent.com/u/1914041?v=3","gravatar_id":"","url":"https://api.github.com/users/fommil","html_url":"https://github.com/fommil","followers_url":"https://api.github.com/users/fommil/followers","following_url":"https://api.github.com/users/fommil/following{/other_user}","gists_url":"https://api.github.com/users/fommil/gists{/gist_id}","starred_url":"https://api.github.com/users/fommil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fommil/subscriptions","organizations_url":"https://api.github.com/users/fommil/orgs","repos_url":"https://api.github.com/users/fommil/repos","events_url":"https://api.github.com/users/fommil/events{/privacy}","received_events_url":"https://api.github.com/users/fommil/received_events","type":"User","site_admin":false},"created_at":"2015-11-01T10:09:13Z","updated_at":"2015-11-01T10:09:13Z","body":"Good news is that Scala 2.12 forces Java 8 so we get a clearer idea of our dependencies eventually."}}