{"action":"created","issue":{"url":"https://api.github.com/repos/EventStore/EventStore/issues/403","labels_url":"https://api.github.com/repos/EventStore/EventStore/issues/403/labels{/name}","comments_url":"https://api.github.com/repos/EventStore/EventStore/issues/403/comments","events_url":"https://api.github.com/repos/EventStore/EventStore/issues/403/events","html_url":"https://github.com/EventStore/EventStore/issues/403","id":59250860,"number":403,"title":"EventStore process starts consuming all available memory and stops responding","user":{"login":"eulerfx","id":609910,"avatar_url":"https://avatars.githubusercontent.com/u/609910?v=3","gravatar_id":"","url":"https://api.github.com/users/eulerfx","html_url":"https://github.com/eulerfx","followers_url":"https://api.github.com/users/eulerfx/followers","following_url":"https://api.github.com/users/eulerfx/following{/other_user}","gists_url":"https://api.github.com/users/eulerfx/gists{/gist_id}","starred_url":"https://api.github.com/users/eulerfx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eulerfx/subscriptions","organizations_url":"https://api.github.com/users/eulerfx/orgs","repos_url":"https://api.github.com/users/eulerfx/repos","events_url":"https://api.github.com/users/eulerfx/events{/privacy}","received_events_url":"https://api.github.com/users/eulerfx/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-02-27T15:31:11Z","updated_at":"2015-04-03T16:49:06Z","closed_at":"2015-03-06T15:52:27Z","body":"I've seen this before when I accidentally swapped the batch size with an offset of about 2 million. EventStore started reading in the batch into memory and stopped responding. This was reported and I think guards are being put in place to prevent reading batches that are too big. This time, I don't think a big batch is being requested, but perhaps there is a huge event somewhere?\r\n\r\nAttached are two snapshots of a PerfView GC Heap analysis:\r\n\r\n![eventstore_perfview_bytes](https://cloud.githubusercontent.com/assets/609910/6415216/a21d2038-be6b-11e4-9505-d461ee402aa2.PNG)\r\n![eventstore_perfview_largeobject](https://cloud.githubusercontent.com/assets/609910/6415217/a3f6d1e2-be6b-11e4-9efd-960fdc630f57.PNG)\r\n"},"comment":{"url":"https://api.github.com/repos/EventStore/EventStore/issues/comments/89350311","html_url":"https://github.com/EventStore/EventStore/issues/403#issuecomment-89350311","issue_url":"https://api.github.com/repos/EventStore/EventStore/issues/403","id":89350311,"user":{"login":"eulerfx","id":609910,"avatar_url":"https://avatars.githubusercontent.com/u/609910?v=3","gravatar_id":"","url":"https://api.github.com/users/eulerfx","html_url":"https://github.com/eulerfx","followers_url":"https://api.github.com/users/eulerfx/followers","following_url":"https://api.github.com/users/eulerfx/following{/other_user}","gists_url":"https://api.github.com/users/eulerfx/gists{/gist_id}","starred_url":"https://api.github.com/users/eulerfx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eulerfx/subscriptions","organizations_url":"https://api.github.com/users/eulerfx/orgs","repos_url":"https://api.github.com/users/eulerfx/repos","events_url":"https://api.github.com/users/eulerfx/events{/privacy}","received_events_url":"https://api.github.com/users/eulerfx/received_events","type":"User","site_admin":false},"created_at":"2015-04-03T16:49:06Z","updated_at":"2015-04-03T16:49:06Z","body":"Same issue again, however this time the server and clients are on v3.0.2. This time, the process `EventStore.ClusterNode` was using about 20 GB of memory and stopped responding. PerfView dump:\r\n\r\n![eventstore_gcdump_perfview2](https://cloud.githubusercontent.com/assets/609910/6985053/f0ec1352-d9fc-11e4-844f-252bedfb716a.PNG)\r\n\r\nAnother strange thing is that when I try to start the eventstore manager after stopping it, it starts the process but it seems to be stuck. It stays at around 30 MB and does nothing. The `System` process on the VM spins at 13% and got into a state where I can't event stop the eventstore manager due to this:\r\n\r\nhttps://social.technet.microsoft.com/Forums/windowsserver/en-US/f0b9bcc1-e127-4c63-b06a-cac9517e1d4b/the-service-cannot-accept-control-messages-at-this-time\r\n\r\nSo I'm not sure if there is an issue with the windows, which is causing the above issue with EventStore in the first place. I don't think it is a huge event as I mentioned before. It could be that something on windows is causing eventstore to back up which in turn causes it to start consuming all the memory. \r\n\r\nCorrect me if I'm wrong, but there is no explicit back-pressure mechanism in eventstore? It just lets the queues fill up? If so, do you think a faulty windows could result in this heap dump?\r\n\r\n\r\n"}}