{"action":"created","issue":{"url":"https://api.github.com/repos/influxdb/influxdb/issues/3253","labels_url":"https://api.github.com/repos/influxdb/influxdb/issues/3253/labels{/name}","comments_url":"https://api.github.com/repos/influxdb/influxdb/issues/3253/comments","events_url":"https://api.github.com/repos/influxdb/influxdb/issues/3253/events","html_url":"https://github.com/influxdb/influxdb/issues/3253","id":93497490,"number":3253,"title":"Influxdb stops writing data after a while","user":{"login":"zeroXten","id":199111,"avatar_url":"https://avatars.githubusercontent.com/u/199111?v=3","gravatar_id":"","url":"https://api.github.com/users/zeroXten","html_url":"https://github.com/zeroXten","followers_url":"https://api.github.com/users/zeroXten/followers","following_url":"https://api.github.com/users/zeroXten/following{/other_user}","gists_url":"https://api.github.com/users/zeroXten/gists{/gist_id}","starred_url":"https://api.github.com/users/zeroXten/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zeroXten/subscriptions","organizations_url":"https://api.github.com/users/zeroXten/orgs","repos_url":"https://api.github.com/users/zeroXten/repos","events_url":"https://api.github.com/users/zeroXten/events{/privacy}","received_events_url":"https://api.github.com/users/zeroXten/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-07-07T10:21:21Z","updated_at":"2015-07-07T10:26:39Z","closed_at":null,"body":"Data is coming in through the graphite plugin. Managed to get 7 hours of data before it stopped writing last night. \r\n\r\nWhen I was debugging it yesterday I had put in some fmt.Printf statements in the graphite parser function. I was tcpdumping port 2003 and watching the printfs. At first they were in sync, then slowly over time the delay between the incoming packet and the printf grew and grew. I could see the influxdb processing blocking on IO, so I rebuilt the box with more processing power, more memory and more IOps. I thought this fixed it but found influxdb to have stopped writing data at 3am.\r\n\r\nYesterday, restarting influxdb seem to help temporarily, but that doesn't seem to be the case anymore.\r\n\r\nThe mod time on the data file is increasing and a dirty diff on a couple of hexdumps shows quite a few changes over a 5 minute period:\r\n\r\n    /var/opt/influxdb/db/graphite_db/default# cp 1 $(date +%s) && sleep 300 && cp 1 $(date +%s)\r\n    /var/opt/influxdb/db/graphite_db/default# diff --suppress-common-lines -y <(hexdump -C 1436261903) <(hexdump -C 1436262213) | wc -l\r\n610095\r\n\r\nI guess I'll try to install the boltdb cli tool to see if I can dig into the data."},"comment":{"url":"https://api.github.com/repos/influxdb/influxdb/issues/comments/119162732","html_url":"https://github.com/influxdb/influxdb/issues/3253#issuecomment-119162732","issue_url":"https://api.github.com/repos/influxdb/influxdb/issues/3253","id":119162732,"user":{"login":"otoolep","id":536312,"avatar_url":"https://avatars.githubusercontent.com/u/536312?v=3","gravatar_id":"","url":"https://api.github.com/users/otoolep","html_url":"https://github.com/otoolep","followers_url":"https://api.github.com/users/otoolep/followers","following_url":"https://api.github.com/users/otoolep/following{/other_user}","gists_url":"https://api.github.com/users/otoolep/gists{/gist_id}","starred_url":"https://api.github.com/users/otoolep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/otoolep/subscriptions","organizations_url":"https://api.github.com/users/otoolep/orgs","repos_url":"https://api.github.com/users/otoolep/repos","events_url":"https://api.github.com/users/otoolep/events{/privacy}","received_events_url":"https://api.github.com/users/otoolep/received_events","type":"User","site_admin":false},"created_at":"2015-07-07T10:26:39Z","updated_at":"2015-07-07T10:26:39Z","body":"@zeroXten -- please tell us the version you are running."}}