{"action":"created","issue":{"url":"https://api.github.com/repos/jemalloc/jemalloc/issues/230","labels_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/230/labels{/name}","comments_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/230/comments","events_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/230/events","html_url":"https://github.com/jemalloc/jemalloc/issues/230","id":74111492,"number":230,"title":"Possible performance regression since 23694b0.","user":{"login":"ldalessa","id":8915711,"avatar_url":"https://avatars.githubusercontent.com/u/8915711?v=3","gravatar_id":"","url":"https://api.github.com/users/ldalessa","html_url":"https://github.com/ldalessa","followers_url":"https://api.github.com/users/ldalessa/followers","following_url":"https://api.github.com/users/ldalessa/following{/other_user}","gists_url":"https://api.github.com/users/ldalessa/gists{/gist_id}","starred_url":"https://api.github.com/users/ldalessa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ldalessa/subscriptions","organizations_url":"https://api.github.com/users/ldalessa/orgs","repos_url":"https://api.github.com/users/ldalessa/repos","events_url":"https://api.github.com/users/ldalessa/events{/privacy}","received_events_url":"https://api.github.com/users/ldalessa/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-05-07T20:12:01Z","updated_at":"2015-05-08T13:36:06Z","closed_at":null,"body":"In [HPX](https://hpx.crest.iu.edu) we make extensive use to manage a number of different address spaces for various kinds of registered memory accesses. Prior to support for explicit caches, we would link two or three instances of libjemalloc with different prefixes, so that we could use caching without having allocations from different address spaces collide in the cache. We have been using 23694b07457f3aaf9605a4ff6b386f3c897eb624 for some time now with good performance results.\r\n\r\nI recently pulled HEAD, (8a03cf039cd06f9fa6972711195055d865673966 as of this writing), and updated our code to use the custom cache capabilities. First of all, this functionality is *wonderful* for us. It has allowed me to simplify a messy allocation infrastructure into something quite a lot more pleasant. We only have to link one libjemalloc(!) and we'll be able to use 4.0 unmodified once it becomes available as long as mallocx is exported.\r\n\r\nI have noticed a performance regression between the two versions though. Part of our system is a user-level-thread scheduler, and one of the SMP torture tests we use is a naive fibonacci that expands the computation tree using threads. This really taxes the allocator—basically it's an allocation stress test. It doesn't use any of the custom allocation infrastructure because we're not including network support in this test.\r\n\r\nIf you download [HPX](https://hpx.crest.iu.edu) and configure/make, you'll get the test in examples/fibonacci, just run it as `fibonnaci --hpx-threads=[threads] n` or use `numactl` to control thread affinity. It should build on x86_64-linux or darwin without any issues. When we preload 23694b07457f3aaf9605a4ff6b386f3c897eb624 vs HEAD, and set MALLOC_CONF=\"lg_chunk:22\", we see that HEAD results in 1.5-2x longer runtimes for the same fibonacci number/scheduler thread count. \r\n\r\nWe're running almost exclusively on NUMA nodes.\r\n\r\n```bash\r\n$ numactl --hardware\r\navailable: 2 nodes (0-1)\r\nnode 0 cpus: 0 2 4 6 8 10 12 14\r\nnode 0 size: 16338 MB\r\nnode 0 free: 13125 MB\r\nnode 1 cpus: 1 3 5 7 9 11 13 15\r\nnode 1 size: 16384 MB\r\nnode 1 free: 14578 MB\r\nnode distances:\r\nnode   0   1 \r\n  0:  10  20 \r\n  1:  20  10 \r\n$ head /proc/cpuinfo \r\nprocessor\t: 0\r\nvendor_id\t: GenuineIntel\r\ncpu family\t: 6\r\nmodel\t\t: 45\r\nmodel name\t: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz\r\nstepping\t: 7\r\nmicrocode\t: 0x70d\r\ncpu MHz\t\t: 2599.875\r\ncache size\t: 20480 KB\r\nphysical id\t: 0\r\n```\r\n\r\nIt's a bit worse on NUMA Haswell systems. I see that there has been *a lot* of great feature and configuration expansion between the two versions. Is there an obvious configuration parameter that would help us here, or is there a real performance regression that needs to be dealt with? The lg_chunk control is vital, with 2^18 it's like 10x slower."},"comment":{"url":"https://api.github.com/repos/jemalloc/jemalloc/issues/comments/100236197","html_url":"https://github.com/jemalloc/jemalloc/issues/230#issuecomment-100236197","issue_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/230","id":100236197,"user":{"login":"mckellyln","id":5116200,"avatar_url":"https://avatars.githubusercontent.com/u/5116200?v=3","gravatar_id":"","url":"https://api.github.com/users/mckellyln","html_url":"https://github.com/mckellyln","followers_url":"https://api.github.com/users/mckellyln/followers","following_url":"https://api.github.com/users/mckellyln/following{/other_user}","gists_url":"https://api.github.com/users/mckellyln/gists{/gist_id}","starred_url":"https://api.github.com/users/mckellyln/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mckellyln/subscriptions","organizations_url":"https://api.github.com/users/mckellyln/orgs","repos_url":"https://api.github.com/users/mckellyln/repos","events_url":"https://api.github.com/users/mckellyln/events{/privacy}","received_events_url":"https://api.github.com/users/mckellyln/received_events","type":"User","site_admin":false},"created_at":"2015-05-08T13:36:06Z","updated_at":"2015-05-08T13:36:06Z","body":"I have noticed a significant slowdown in jemalloc performance since a few revs ago as well.  I did not record what the specific version was before that I used, sorry,  but the latest version is at least 2x+ slower in our tests, both single and many threads.  Nothing I can find in the test code has changed which made me think it was something in jemalloc but I have not instrumented/profiled to find out any more info."}}