{"action":"created","issue":{"url":"https://api.github.com/repos/labrad/labrad/issues/12","labels_url":"https://api.github.com/repos/labrad/labrad/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/labrad/labrad/issues/12/comments","events_url":"https://api.github.com/repos/labrad/labrad/issues/12/events","html_url":"https://github.com/labrad/labrad/issues/12","id":114935599,"number":12,"title":"Real time plotting. Any recommendations? ","user":{"login":"zibrov-zlobin","id":3323701,"avatar_url":"https://avatars.githubusercontent.com/u/3323701?v=3","gravatar_id":"","url":"https://api.github.com/users/zibrov-zlobin","html_url":"https://github.com/zibrov-zlobin","followers_url":"https://api.github.com/users/zibrov-zlobin/followers","following_url":"https://api.github.com/users/zibrov-zlobin/following{/other_user}","gists_url":"https://api.github.com/users/zibrov-zlobin/gists{/gist_id}","starred_url":"https://api.github.com/users/zibrov-zlobin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zibrov-zlobin/subscriptions","organizations_url":"https://api.github.com/users/zibrov-zlobin/orgs","repos_url":"https://api.github.com/users/zibrov-zlobin/repos","events_url":"https://api.github.com/users/zibrov-zlobin/events{/privacy}","received_events_url":"https://api.github.com/users/zibrov-zlobin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-11-03T23:36:02Z","updated_at":"2015-11-24T03:13:37Z","closed_at":null,"body":"I'm sure you've thought of all the possible solutions to how to best organize real-time plotting, so I was wondering if anyone could give me some tips.\r\nI need to plot  time series and possibly 2d heat plots in real time. \r\n\r\nAs I understand, the way to do it would be:\r\n\r\nThe Client retrieves data from the devices and sends it off to the Data Vault server. This creates/ updates a csv file. The data vault on the other hand at the event of new data received sends a signal to all other listening servers/clients and particularly some plotting client. The plotter then reads the csv file and append the last row  to the graph. \r\n\r\nIf the dataset becomes large, wouldn't this pose speed issues, since the whole csv file needs to be read in order to update the graph? \r\n\r\nAm I missing something? \r\n\r\nOr the scheme to do would be to have a data_plotting server that receives data from the client without ever querying the data_Vault? \r\nWould it be beneficial to use some sort of TSDB for this? Say influxdb + graphite/graphana? Maybe something working already exists? \r\n\r\nThanks,\r\nSasha  \r\n"},"comment":{"url":"https://api.github.com/repos/labrad/labrad/issues/comments/159138813","html_url":"https://github.com/labrad/labrad/issues/12#issuecomment-159138813","issue_url":"https://api.github.com/repos/labrad/labrad/issues/12","id":159138813,"user":{"login":"DanielSank","id":4584456,"avatar_url":"https://avatars.githubusercontent.com/u/4584456?v=3","gravatar_id":"","url":"https://api.github.com/users/DanielSank","html_url":"https://github.com/DanielSank","followers_url":"https://api.github.com/users/DanielSank/followers","following_url":"https://api.github.com/users/DanielSank/following{/other_user}","gists_url":"https://api.github.com/users/DanielSank/gists{/gist_id}","starred_url":"https://api.github.com/users/DanielSank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DanielSank/subscriptions","organizations_url":"https://api.github.com/users/DanielSank/orgs","repos_url":"https://api.github.com/users/DanielSank/repos","events_url":"https://api.github.com/users/DanielSank/events{/privacy}","received_events_url":"https://api.github.com/users/DanielSank/received_events","type":"User","site_admin":false},"created_at":"2015-11-24T03:13:37Z","updated_at":"2015-11-24T03:13:37Z","body":"> The Client retrieves data from the devices and sends it off to the Data Vault server. This creates/ updates a csv file. The data vault on the other hand at the event of new data received sends a signal to all other listening servers/clients and particularly some plotting client. The plotter then reads the csv file and append the last row to the graph.\r\n\r\nThat's exactly how we do it.\r\n\r\n> If the dataset becomes large, wouldn't this pose speed issues, since the whole csv file needs to be read in order to update the graph?\r\n\r\nWell, yes, large data sets take a few moments to load. We have client-side wrappers representing a data set which manage fetching data in reasonable chunks, etc.\r\n\r\n> Am I missing something?\r\n\r\nDoesn't sound like it.\r\n\r\n> Or the scheme to do would be to have a data_plotting server that receives data from the client without ever querying the data_Vault?\r\n\r\nI suppose that's possible, but it hasn't been necessary for us.\r\n\r\nWe use a combination of tools for plotting. Our most time honored one is a legacy program written in Delphi. We are phasing that out in favor of a web-based system (which functions almost exactly as you've proposed). We have also used PyQT with the associated matplotlib plug-ins for logging cryostat temperature, pressure, etc.\r\n"}}