{"action":"created","issue":{"url":"https://api.github.com/repos/sma-wideband/sdbe/issues/11","labels_url":"https://api.github.com/repos/sma-wideband/sdbe/issues/11/labels{/name}","comments_url":"https://api.github.com/repos/sma-wideband/sdbe/issues/11/comments","events_url":"https://api.github.com/repos/sma-wideband/sdbe/issues/11/events","html_url":"https://github.com/sma-wideband/sdbe/issues/11","id":76468622,"number":11,"title":"Cooperative data processing pipeline framework","user":{"login":"ruriktherus","id":372037,"avatar_url":"https://avatars.githubusercontent.com/u/372037?v=3","gravatar_id":"","url":"https://api.github.com/users/ruriktherus","html_url":"https://github.com/ruriktherus","followers_url":"https://api.github.com/users/ruriktherus/followers","following_url":"https://api.github.com/users/ruriktherus/following{/other_user}","gists_url":"https://api.github.com/users/ruriktherus/gists{/gist_id}","starred_url":"https://api.github.com/users/ruriktherus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ruriktherus/subscriptions","organizations_url":"https://api.github.com/users/ruriktherus/orgs","repos_url":"https://api.github.com/users/ruriktherus/repos","events_url":"https://api.github.com/users/ruriktherus/events{/privacy}","received_events_url":"https://api.github.com/users/ruriktherus/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ruriktherus","id":372037,"avatar_url":"https://avatars.githubusercontent.com/u/372037?v=3","gravatar_id":"","url":"https://api.github.com/users/ruriktherus","html_url":"https://github.com/ruriktherus","followers_url":"https://api.github.com/users/ruriktherus/followers","following_url":"https://api.github.com/users/ruriktherus/following{/other_user}","gists_url":"https://api.github.com/users/ruriktherus/gists{/gist_id}","starred_url":"https://api.github.com/users/ruriktherus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ruriktherus/subscriptions","organizations_url":"https://api.github.com/users/ruriktherus/orgs","repos_url":"https://api.github.com/users/ruriktherus/repos","events_url":"https://api.github.com/users/ruriktherus/events{/privacy}","received_events_url":"https://api.github.com/users/ruriktherus/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2015-05-14T19:43:05Z","updated_at":"2015-05-14T21:16:11Z","closed_at":null,"body":"We need to develop a pipeline framework to process the data in discrete steps which must have:\r\n\r\n* Modularity\r\n* Low overhead\r\n* Parallel instances \r\n* Nest-able steps\r\n\r\nAnd ideally will have:\r\n\r\n* Branch-able steps\r\n* Statistics tracking\r\n"},"comment":{"url":"https://api.github.com/repos/sma-wideband/sdbe/issues/comments/102171439","html_url":"https://github.com/sma-wideband/sdbe/issues/11#issuecomment-102171439","issue_url":"https://api.github.com/repos/sma-wideband/sdbe/issues/11","id":102171439,"user":{"login":"david-macmahon","id":730406,"avatar_url":"https://avatars.githubusercontent.com/u/730406?v=3","gravatar_id":"","url":"https://api.github.com/users/david-macmahon","html_url":"https://github.com/david-macmahon","followers_url":"https://api.github.com/users/david-macmahon/followers","following_url":"https://api.github.com/users/david-macmahon/following{/other_user}","gists_url":"https://api.github.com/users/david-macmahon/gists{/gist_id}","starred_url":"https://api.github.com/users/david-macmahon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/david-macmahon/subscriptions","organizations_url":"https://api.github.com/users/david-macmahon/orgs","repos_url":"https://api.github.com/users/david-macmahon/repos","events_url":"https://api.github.com/users/david-macmahon/events{/privacy}","received_events_url":"https://api.github.com/users/david-macmahon/received_events","type":"User","site_admin":false},"created_at":"2015-05-14T21:16:11Z","updated_at":"2015-05-14T21:16:11Z","body":"One possibility is HASHPIPE:\n\nhttps://github.com/david-macmahon/hashpipe\n\nThis is is the framework that what we use for PAPER and SERENDIP6, both GPU pipelines that get data via 10 GbE from ROACH2s.\n\nI recently added support for \"packet sockets\" which is lower level but better performing than \"IP sockets\".\n\nIt's not the most heavily documented framework, but it works quite well for our applications.  It is based on the VEGAS spectrometer code (which was based on the GUPPI code), but HASHPIPE adds a bunch of new and useful features and separates the \"framework\" from the \"application\".\n\nThe same HASHPIPE installation can run different \"plug-ins\".  Here are links to the repos for the PAPER and SERENDIP6 plug-ins:\n\nhttps://github.com/david-macmahon/paper_gpu\n\nhttp://astro.berkeley.edu/~davidm/serendip6.git\n\nDave\n\nOn May 14, 2015, at 12:43 PM, Rurik A. Primiani wrote:\n\n> We need to develop a pipeline framework to process the data in discrete steps which must have:\n> \n> \t� Modularity\n> \t� Low overhead\n> \t� Parallel instances\n> \t� Nest-able steps\n> And ideally will have:\n> \n> \t� Branch-able steps\n> \t� Statistics tracking\n> �\n> Reply to this email directly or view it on GitHub.\n> \n\n"}}