{"action":"opened","issue":{"url":"https://api.github.com/repos/stan-dev/stan/issues/1321","labels_url":"https://api.github.com/repos/stan-dev/stan/issues/1321/labels{/name}","comments_url":"https://api.github.com/repos/stan-dev/stan/issues/1321/comments","events_url":"https://api.github.com/repos/stan-dev/stan/issues/1321/events","html_url":"https://github.com/stan-dev/stan/issues/1321","id":59417476,"number":1321,"title":"Tweak format of written .csv files","user":{"login":"bgoodri","id":3386282,"avatar_url":"https://avatars.githubusercontent.com/u/3386282?v=3","gravatar_id":"","url":"https://api.github.com/users/bgoodri","html_url":"https://github.com/bgoodri","followers_url":"https://api.github.com/users/bgoodri/followers","following_url":"https://api.github.com/users/bgoodri/following{/other_user}","gists_url":"https://api.github.com/users/bgoodri/gists{/gist_id}","starred_url":"https://api.github.com/users/bgoodri/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bgoodri/subscriptions","organizations_url":"https://api.github.com/users/bgoodri/orgs","repos_url":"https://api.github.com/users/bgoodri/repos","events_url":"https://api.github.com/users/bgoodri/events{/privacy}","received_events_url":"https://api.github.com/users/bgoodri/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-03-01T21:07:15Z","updated_at":"2015-03-01T21:07:15Z","closed_at":null,"body":"The current output of the .csv files when you include the warmup period looks like\r\n```\r\n# lines\r\n# of\r\n...\r\n# metadata\r\nlp__,accept_stat__,...\r\nnumber,number,...\r\nnumber,number,...\r\n...\r\nnumber,number,...\r\n# Adaptation terminated\r\n# Step size = number\r\n# Diagonal elements of inverse mass matrix:\r\n# number, number, ...\r\nnumber,number,...\r\nnumber,number,...\r\n...\r\nnumber,number,...\r\n# Elapsed Time: number seconds (Warm-up)\r\n#               number seconds (Sampling)\r\n#               number seconds (Total)\r\n```\r\nThis format is inefficient in time and RAM to read (e.g. into R) because it has to scan each line as text for the leading pound sign. Also, it does not play nicely with the optimized `fread` function in the data.table package that was apparently used by \r\nhttps://groups.google.com/d/msg/stan-users/urZeEMjjPHM/R0S41-hQbPMJ\r\n\r\nI think it would be better to \r\n\r\n1. Put a blank line in the .csv file after the last iteration\r\n2. Put the four lines of adaptation info after that instead of in the middle of the file\r\n\r\nThus, the format would look like\r\n```\r\n# lines\r\n# of\r\n...\r\n# metadata\r\nlp__,accept_stat__,...\r\nnumber,number,...\r\nnumber,number,...\r\n...\r\nnumber,number,...\r\n\r\n# Adaptation terminated\r\n# Step size = number\r\n# Diagonal elements of inverse mass matrix:\r\n# number, number, ...\r\n# Elapsed Time: number seconds (Warm-up)\r\n#               number seconds (Sampling)\r\n#               number seconds (Total)\r\n```\r\nDoes anything depend on the absence of blank lines or the presence of the adaptation info being in between the warmup iterations and the sampled iterations? "}}