{"action":"created","issue":{"url":"https://api.github.com/repos/jimhigson/oboe.js/issues/68","labels_url":"https://api.github.com/repos/jimhigson/oboe.js/issues/68/labels{/name}","comments_url":"https://api.github.com/repos/jimhigson/oboe.js/issues/68/comments","events_url":"https://api.github.com/repos/jimhigson/oboe.js/issues/68/events","html_url":"https://github.com/jimhigson/oboe.js/issues/68","id":70216794,"number":68,"title":"Memory leaks with oboe.drop","user":{"login":"goloroden","id":906327,"avatar_url":"https://avatars.githubusercontent.com/u/906327?v=3","gravatar_id":"","url":"https://api.github.com/users/goloroden","html_url":"https://github.com/goloroden","followers_url":"https://api.github.com/users/goloroden/followers","following_url":"https://api.github.com/users/goloroden/following{/other_user}","gists_url":"https://api.github.com/users/goloroden/gists{/gist_id}","starred_url":"https://api.github.com/users/goloroden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/goloroden/subscriptions","organizations_url":"https://api.github.com/users/goloroden/orgs","repos_url":"https://api.github.com/users/goloroden/repos","events_url":"https://api.github.com/users/goloroden/events{/privacy}","received_events_url":"https://api.github.com/users/goloroden/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-04-22T19:39:18Z","updated_at":"2015-08-28T14:42:27Z","closed_at":null,"body":"I have seen that there already was an issue on memory leaks (it's #45), and that things should be resolved by returning `oboe.drop` as [documented here](http://oboejs.com/api#dropping-nodes).\r\n\r\nUnfortunately, 2.1.1 seems to still have memory issues (or I am getting it absolutely wrong how to use `oboe.drop` correctly).\r\n\r\nMy setup is as follows: I have a server based on Express that delivers an endless JSON array, and I have a client that uses Oboe to stream these data. I measure the memory consumption of this client, and within a few hours it uses hundreds of MBytes, and GC obviously does not clean up as expected.\r\n\r\nThe server looks like this:\r\n\r\n```javascript\r\nvar http = require('http');\r\n\r\nvar express = require('express');\r\n\r\nvar app = express();\r\n\r\napp.get('/', function (req, res) {\r\n  var i = 0;\r\n\r\n  var sendEventToClient = function (event) {\r\n    res.write(JSON.stringify(event) + ',');\r\n  };\r\n\r\n  req.setTimeout(0);\r\n  res.setTimeout(0);\r\n\r\n  res.writeHead(200, {\r\n    'content-type': 'application/json'\r\n  });\r\n  res.write('[');\r\n\r\n  setInterval(function () {\r\n    sendEventToClient({\r\n      foo: 'bar',\r\n      bar: 'baz',\r\n      nufta: 23,\r\n      counter: i++\r\n    });\r\n  }, 10);\r\n});\r\n\r\nhttp.createServer(app).listen(3000);\r\n```\r\n\r\nThe client looks like this:\r\n\r\n```javascript\r\nvar url = require('url');\r\n\r\nvar oboe = require('oboe'),\r\n    Stethoskop = require('stethoskop');\r\n\r\nvar stethoskop = new Stethoskop({\r\n  from: {\r\n    application: 'client.js'\r\n  },\r\n  to: {\r\n    host: 'localhost',\r\n    port: 8125\r\n  },\r\n  enabled: true\r\n});\r\n\r\noboe({\r\n  url: url.format({\r\n    protocol: 'http',\r\n    hostname: 'localhost',\r\n    port: 3000,\r\n    pathname: '/'\r\n  }),\r\n  cached: false\r\n}).on('node:!.*', function (event) {\r\n  console.log(event);\r\n  return oboe.drop;\r\n}).on('fail', function (err) {\r\n  console.log('Error!', err.thrown);\r\n});\r\n```\r\n\r\nSo, the server sends a new object every 10 ms, and the client should do nothing with it but drop it. I am using [stethoskop](https://www.npmjs.com/package/stethoskop) to measure the client's fitness: It sends the CPU and memory data to a StatsD server.\r\n\r\nI have run these two processes for 4.5 hours, and the memory consumption looks like this:\r\n\r\n![graphite](https://cloud.githubusercontent.com/assets/906327/7283606/0010597a-e938-11e4-89c4-85895ff9995b.png)\r\n\r\nI also tried to run it returning `null` instead of `oboe.drop`, same result (the left part is the same test as above, the right part is with `null`, so basically both options show the very same behavior. The drop to 0 in the middle is **not** because of GC, it's because I stopped and restarted the processes):\r\n\r\n![test 2](https://cloud.githubusercontent.com/assets/906327/7291144/0a8e5934-e98c-11e4-8b37-21ad7dbc1370.png)\r\n\r\nSo, to cut a long story short, basically I have two questions:\r\n\r\n- Am I doing something wrong in the client? If so, what do I need to correct? Any hints?\r\n- If not, this seems to be a bug in Oboe. Can I do something to help fix it? Pointing out and giving some hints where to look might already be quite helpful :-)\r\n\r\nMoreover, the [documentation](http://oboejs.com/api#dropping-nodes) states:\r\n\r\n> Dropping from an array will result in an array with holes in it. This is intentional so that the array indices from the original JSON are preserved:\r\n\r\nMy guess is that dropping nodes only works (wrt memory consumption) for objects, but not for arrays. Since I am using an array as outer container here, and since sparse arrays *seem* to capture more memory than dense ones, this might be the cause of the problem. Please note that I'm not [too sure](https://stackoverflow.com/questions/29814940/memory-consumption-of-sparse-arrays-in-node-js) about memory behavior of sparse arrays, so any answer in this direction will be appreciated."},"comment":{"url":"https://api.github.com/repos/jimhigson/oboe.js/issues/comments/135793107","html_url":"https://github.com/jimhigson/oboe.js/issues/68#issuecomment-135793107","issue_url":"https://api.github.com/repos/jimhigson/oboe.js/issues/68","id":135793107,"user":{"login":"JuanCaicedo","id":4341496,"avatar_url":"https://avatars.githubusercontent.com/u/4341496?v=3","gravatar_id":"","url":"https://api.github.com/users/JuanCaicedo","html_url":"https://github.com/JuanCaicedo","followers_url":"https://api.github.com/users/JuanCaicedo/followers","following_url":"https://api.github.com/users/JuanCaicedo/following{/other_user}","gists_url":"https://api.github.com/users/JuanCaicedo/gists{/gist_id}","starred_url":"https://api.github.com/users/JuanCaicedo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuanCaicedo/subscriptions","organizations_url":"https://api.github.com/users/JuanCaicedo/orgs","repos_url":"https://api.github.com/users/JuanCaicedo/repos","events_url":"https://api.github.com/users/JuanCaicedo/events{/privacy}","received_events_url":"https://api.github.com/users/JuanCaicedo/received_events","type":"User","site_admin":false},"created_at":"2015-08-28T14:42:27Z","updated_at":"2015-08-28T14:42:27Z","body":"@egervari Have you tried the JSON stringify/parse trick that @badisa did? It would be interesting to see if that works"}}