{"action":"created","issue":{"url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/64","labels_url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/64/comments","events_url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/64/events","html_url":"https://github.com/BioJulia/Bio.jl/issues/64","id":92953630,"number":64,"title":"RFC: Bioinformatics project management with BioJulia","user":{"login":"Ward9250","id":3399309,"avatar_url":"https://avatars.githubusercontent.com/u/3399309?v=3","gravatar_id":"","url":"https://api.github.com/users/Ward9250","html_url":"https://github.com/Ward9250","followers_url":"https://api.github.com/users/Ward9250/followers","following_url":"https://api.github.com/users/Ward9250/following{/other_user}","gists_url":"https://api.github.com/users/Ward9250/gists{/gist_id}","starred_url":"https://api.github.com/users/Ward9250/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ward9250/subscriptions","organizations_url":"https://api.github.com/users/Ward9250/orgs","repos_url":"https://api.github.com/users/Ward9250/repos","events_url":"https://api.github.com/users/Ward9250/events{/privacy}","received_events_url":"https://api.github.com/users/Ward9250/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-07-04T00:31:34Z","updated_at":"2015-07-04T15:04:39Z","closed_at":null,"body":"This is to further my idea articulated on Gitter. As the core library becomes even more complete, and as the BioJulia organisation gets more attention, one of the things I think would help set BioJulia apart from other Bio* projects (besides being fast and making the best use of Julia's features), would be if we also made it very easy for casual scripters (By that I mean lab based biologists that have to then turn to do some bioinformatics at the end of their experiment) to manage their Bioinformatics projects: if you're already scripting a few data processing steps with BioJulia tools, why not also manage the project with BioJulia?\r\n\r\nWhat I think would be great to see, is being able to start a Bioinformatics project directory from Julia, which is reproducible and self contained, and so perhaps makes use of its own package repo. It could also make use of both Git and Dat to A). Record code changes to scripts and pipelines in a project, and B). keep track and version data produced by scripts in your Bioinformatics project. With regards to pipelines, the ability to scrip pipelines that make use of Bio.jl structures and algorithms, with a streams/flow-based programming approach would be awesome. This all tied together would make it awesome to manage and run a reproducible Bioinformatics project with BioJulia.\r\n\r\nSo breaking it down a bit, I guess what I'm saying with this brain dump is:\r\n\r\n* A module for interfacing - reading and writing to and from Dat.\r\n* Streams / Flow-based programming (ideally that also makes use of multiple processes - I think nextflow does this).\r\n*   Project based packages like packrat for R or virtualenv for Python.\r\n\r\nWould be cool milestones."},"comment":{"url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/comments/118518453","html_url":"https://github.com/BioJulia/Bio.jl/issues/64#issuecomment-118518453","issue_url":"https://api.github.com/repos/BioJulia/Bio.jl/issues/64","id":118518453,"user":{"login":"Blahah","id":836040,"avatar_url":"https://avatars.githubusercontent.com/u/836040?v=3","gravatar_id":"","url":"https://api.github.com/users/Blahah","html_url":"https://github.com/Blahah","followers_url":"https://api.github.com/users/Blahah/followers","following_url":"https://api.github.com/users/Blahah/following{/other_user}","gists_url":"https://api.github.com/users/Blahah/gists{/gist_id}","starred_url":"https://api.github.com/users/Blahah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Blahah/subscriptions","organizations_url":"https://api.github.com/users/Blahah/orgs","repos_url":"https://api.github.com/users/Blahah/repos","events_url":"https://api.github.com/users/Blahah/events{/privacy}","received_events_url":"https://api.github.com/users/Blahah/received_events","type":"User","site_admin":false},"created_at":"2015-07-04T15:04:39Z","updated_at":"2015-07-04T15:04:39Z","body":"Great about the dat package - push up to BioJulia?\r\n\r\nI like the idea of a project management module proving the higher level functionality. Expanding on your `project_save()`, I'm thinking of a workflow like...\r\n\r\n```\r\nproject_init('dirname') # create dirname directory, git init and dat init\r\n# OR\r\nproject_init('dirname', template=true) # same but using template layout\r\n\r\n# do some work\r\n\r\nproject_save('description of the changes') # git commit changes, dat import changes, store git and dat IDs\r\nproject_log() # print log of the code and data changes\r\nproject_checkout('someid') # checkout a previous state\r\nproject_checkout() # checkout the HEAD\r\n\r\nproject_version('version number', 'description') # tag a version of the project\r\n```\r\n\r\nPotentially this idea could be extended to include other LIMS-like features, such as tracking of sequencing samples and metadata, and the ability to automate publishing data to ENA/SRA, analysis to github/zenodo, etc."}}