{"action":"created","issue":{"url":"https://api.github.com/repos/kripken/emscripten/issues/3253","labels_url":"https://api.github.com/repos/kripken/emscripten/issues/3253/labels{/name}","comments_url":"https://api.github.com/repos/kripken/emscripten/issues/3253/comments","events_url":"https://api.github.com/repos/kripken/emscripten/issues/3253/events","html_url":"https://github.com/kripken/emscripten/issues/3253","id":61282462,"number":3253,"title":"Support for large memory handling?","user":{"login":"evanw","id":406394,"avatar_url":"https://avatars.githubusercontent.com/u/406394?v=3","gravatar_id":"","url":"https://api.github.com/users/evanw","html_url":"https://github.com/evanw","followers_url":"https://api.github.com/users/evanw/followers","following_url":"https://api.github.com/users/evanw/following{/other_user}","gists_url":"https://api.github.com/users/evanw/gists{/gist_id}","starred_url":"https://api.github.com/users/evanw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evanw/subscriptions","organizations_url":"https://api.github.com/users/evanw/orgs","repos_url":"https://api.github.com/users/evanw/repos","events_url":"https://api.github.com/users/evanw/events{/privacy}","received_events_url":"https://api.github.com/users/evanw/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-13T22:36:27Z","updated_at":"2015-03-14T03:02:07Z","closed_at":null,"body":"The memory system that emscripten uses seems poorly suited for managing large amounts of memory. Dynamic growth via enlargeMemory() has several undesirable properties:\r\n\r\n* If memory growth is disabled, it aborts immediately. This prevents our code from handling out-of-memory errors by checking for NULL from malloc().\r\n* If memory growth is enabled, it causes an infinite loop after growing past 1gb (it tries to grow memory to 1 << 31 bytes which is negative). This is obviously a bad user experience. Ideally it would grow from 1gb to something around 1.9gb if possible, otherwise it would return NULL.\r\n* When the reserved memory is exhausted, we would like the chance to free some of our cached data first and only then call enlargeMemory() if there still isn't enough memory.\r\n\r\nHow have other people been dealing with these issues? Should we be compiling our own malloc() implementation? If so, how easy is this to integrate with the rest of emscripten?\r\n"},"comment":{"url":"https://api.github.com/repos/kripken/emscripten/issues/comments/79754510","html_url":"https://github.com/kripken/emscripten/issues/3253#issuecomment-79754510","issue_url":"https://api.github.com/repos/kripken/emscripten/issues/3253","id":79754510,"user":{"login":"kripken","id":173661,"avatar_url":"https://avatars.githubusercontent.com/u/173661?v=3","gravatar_id":"","url":"https://api.github.com/users/kripken","html_url":"https://github.com/kripken","followers_url":"https://api.github.com/users/kripken/followers","following_url":"https://api.github.com/users/kripken/following{/other_user}","gists_url":"https://api.github.com/users/kripken/gists{/gist_id}","starred_url":"https://api.github.com/users/kripken/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kripken/subscriptions","organizations_url":"https://api.github.com/users/kripken/orgs","repos_url":"https://api.github.com/users/kripken/repos","events_url":"https://api.github.com/users/kripken/events{/privacy}","received_events_url":"https://api.github.com/users/kripken/received_events","type":"User","site_admin":false},"created_at":"2015-03-14T03:02:07Z","updated_at":"2015-03-14T03:02:07Z","body":"I think most people are not using heaps of that size. For example evan Unreal strongly recommends people use a 512MB heap and no larger, because browsers often can't load any bigger.\r\n\r\n * We could add an option for malloc to be fallible, if that would be useful.\r\n * We should fix growing to get closer to the current 2GB limit.\r\n * Do native libcs have any functionality like that? I'd like to use something standard if there is. But, it would be trivial to add - our `sbrk()` could call into `Module.memoryPressureWarning()` or such.\r\n * You can also compile in your own malloc. Just build it and it will override the system malloc/free. You just need to make sure it calls `sbrk()` properly. But, we should just fix the above 3 issues for everyone, I don't think people need a custom malloc for this (might need one for other reasons of course).\r\n"}}