{"action":"created","issue":{"url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/45","labels_url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/45/comments","events_url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/45/events","html_url":"https://github.com/googlegenomics/spark-examples/issues/45","id":46121862,"number":45,"title":"Framework for GoogleGenomics/Spark with a CLI and Pipelines","user":{"login":"pgrosu","id":6555937,"avatar_url":"https://avatars.githubusercontent.com/u/6555937?v=3","gravatar_id":"","url":"https://api.github.com/users/pgrosu","html_url":"https://github.com/pgrosu","followers_url":"https://api.github.com/users/pgrosu/followers","following_url":"https://api.github.com/users/pgrosu/following{/other_user}","gists_url":"https://api.github.com/users/pgrosu/gists{/gist_id}","starred_url":"https://api.github.com/users/pgrosu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pgrosu/subscriptions","organizations_url":"https://api.github.com/users/pgrosu/orgs","repos_url":"https://api.github.com/users/pgrosu/repos","events_url":"https://api.github.com/users/pgrosu/events{/privacy}","received_events_url":"https://api.github.com/users/pgrosu/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2014-10-17T17:10:00Z","updated_at":"2015-01-30T21:18:18Z","closed_at":"2015-01-30T21:18:18Z","body":"Hi Elmer and Nicole,\r\n\r\nWhat about creating a framework for GoogleGenomics/Spark, and then one can connect via a command-line interface and launch different tasks via classes.  Ideally these will also have the ability to be pipelined together.  \r\n\r\nI am thinking about something along the lines of Weka:\r\n\r\nhttp://www.cs.waikato.ac.nz/ml/weka/\r\n\r\nYou can perform command-line interface (CLI) like this: \r\n\r\nhttp://weka.wikispaces.com/Primer\r\n\r\nYou can even have a pipeline like this: \r\n\r\nhttp://weka.wikispaces.com/Exporting+Charts+from+the+Knowledge+Flow\r\n\r\nMaybe you can integrate with Google's Dataflow on this, based on what Cassie mentioned recently.  With a framework you can easily add then a class for caching out the similarity matrices, and then just reloading them as necessary. \r\n\r\nPaul"},"comment":{"url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/comments/72271880","html_url":"https://github.com/googlegenomics/spark-examples/issues/45#issuecomment-72271880","issue_url":"https://api.github.com/repos/googlegenomics/spark-examples/issues/45","id":72271880,"user":{"login":"deflaux","id":1395500,"avatar_url":"https://avatars.githubusercontent.com/u/1395500?v=3","gravatar_id":"","url":"https://api.github.com/users/deflaux","html_url":"https://github.com/deflaux","followers_url":"https://api.github.com/users/deflaux/followers","following_url":"https://api.github.com/users/deflaux/following{/other_user}","gists_url":"https://api.github.com/users/deflaux/gists{/gist_id}","starred_url":"https://api.github.com/users/deflaux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deflaux/subscriptions","organizations_url":"https://api.github.com/users/deflaux/orgs","repos_url":"https://api.github.com/users/deflaux/repos","events_url":"https://api.github.com/users/deflaux/events{/privacy}","received_events_url":"https://api.github.com/users/deflaux/received_events","type":"User","site_admin":false},"created_at":"2015-01-30T21:18:18Z","updated_at":"2015-01-30T21:18:18Z","body":"Its all getting closer http://blog.cloudera.com/blog/2015/01/new-in-cloudera-labs-google-cloud-dataflow-on-apache-spark/\r\n\r\nThanks Paul!"}}