{"action":"created","issue":{"url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/117","labels_url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/117/labels{/name}","comments_url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/117/comments","events_url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/117/events","html_url":"https://github.com/aspnet/KestrelHttpServer/issues/117","id":91878342,"number":117,"title":"Thoughts for SocketOutput fine-tuning","user":{"login":"lodejard","id":1693989,"avatar_url":"https://avatars.githubusercontent.com/u/1693989?v=3","gravatar_id":"","url":"https://api.github.com/users/lodejard","html_url":"https://github.com/lodejard","followers_url":"https://api.github.com/users/lodejard/followers","following_url":"https://api.github.com/users/lodejard/following{/other_user}","gists_url":"https://api.github.com/users/lodejard/gists{/gist_id}","starred_url":"https://api.github.com/users/lodejard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lodejard/subscriptions","organizations_url":"https://api.github.com/users/lodejard/orgs","repos_url":"https://api.github.com/users/lodejard/repos","events_url":"https://api.github.com/users/lodejard/events{/privacy}","received_events_url":"https://api.github.com/users/lodejard/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/aspnet/KestrelHttpServer/labels/1%20-%20Ready","name":"1 - Ready","color":"CCCCCC"},{"url":"https://api.github.com/repos/aspnet/KestrelHttpServer/labels/perf","name":"perf","color":"fef2c0"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/aspnet/KestrelHttpServer/milestones/14","html_url":"https://github.com/aspnet/KestrelHttpServer/milestones/1.0.0-rc1","labels_url":"https://api.github.com/repos/aspnet/KestrelHttpServer/milestones/14/labels","id":1256011,"number":14,"title":"1.0.0-rc1","description":"","creator":{"login":"DamianEdwards","id":249088,"avatar_url":"https://avatars.githubusercontent.com/u/249088?v=3","gravatar_id":"","url":"https://api.github.com/users/DamianEdwards","html_url":"https://github.com/DamianEdwards","followers_url":"https://api.github.com/users/DamianEdwards/followers","following_url":"https://api.github.com/users/DamianEdwards/following{/other_user}","gists_url":"https://api.github.com/users/DamianEdwards/gists{/gist_id}","starred_url":"https://api.github.com/users/DamianEdwards/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DamianEdwards/subscriptions","organizations_url":"https://api.github.com/users/DamianEdwards/orgs","repos_url":"https://api.github.com/users/DamianEdwards/repos","events_url":"https://api.github.com/users/DamianEdwards/events{/privacy}","received_events_url":"https://api.github.com/users/DamianEdwards/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":0,"state":"open","created_at":"2015-08-14T23:12:49Z","updated_at":"2015-08-18T18:05:03Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2015-06-29T18:41:06Z","updated_at":"2015-08-19T20:41:01Z","closed_at":null,"body":"Imagine the state of the SocketOutput had a \"buffers pending\" list and a \"writes sending\" count. \r\n\r\nA buffer in this case would be an object that has an IntPtr to native memory, the actual length of that native memory, the offset of active data in the buffer, and the length of active data in the buffer.\r\n\r\nThe actual IntPtr and length of native memory would come from a pool. That pool would VirtualAlloc a larger contiguous space at once, and hand out references to buffers which are aligned and sized on page boundaries (4k).\r\n\r\nWhen a Write comes into the SocketOutput, if there are \"buffers pending\" and the final buffer has trailing space between the end of active data and end of native memory, it would first copy as much of the outgoing data (possibly all) into the existing buffer. If there are more bytes to send which have not been copied, a new 4k buffer is acquired from the pool and the process is repeated until all bytes to send are copied to \"buffers pending\"\r\n\r\nIf at the end of this process the \"writes sending\" count is less than three, increment it, and Post a callback on the uv thread. In that callback, all buffers are removed from the \"buffers pending\" list and sent to uv. When the uv write callback is invoked then the buffers which were written are returned to the pool. Then, if there are \"buffers pending\" remove them all and send to uv. In either, if a \"buffers pending\" check reveals that list is empty, decrement the \"writes sending\" count.\r\n\r\nThe SocketOutput will need a simple lock object at both of these points to ensure the data structures aren't compromised."},"comment":{"url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/comments/132776359","html_url":"https://github.com/aspnet/KestrelHttpServer/issues/117#issuecomment-132776359","issue_url":"https://api.github.com/repos/aspnet/KestrelHttpServer/issues/117","id":132776359,"user":{"login":"benaadams","id":1142958,"avatar_url":"https://avatars.githubusercontent.com/u/1142958?v=3","gravatar_id":"","url":"https://api.github.com/users/benaadams","html_url":"https://github.com/benaadams","followers_url":"https://api.github.com/users/benaadams/followers","following_url":"https://api.github.com/users/benaadams/following{/other_user}","gists_url":"https://api.github.com/users/benaadams/gists{/gist_id}","starred_url":"https://api.github.com/users/benaadams/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benaadams/subscriptions","organizations_url":"https://api.github.com/users/benaadams/orgs","repos_url":"https://api.github.com/users/benaadams/repos","events_url":"https://api.github.com/users/benaadams/events{/privacy}","received_events_url":"https://api.github.com/users/benaadams/received_events","type":"User","site_admin":false},"created_at":"2015-08-19T20:41:01Z","updated_at":"2015-08-19T20:41:01Z","body":"> Pre-allocating over 64k per thread on startup might be a bit heavy, though I agree with the idea of loop-memory affinity.\r\n\r\nIf you have a thread per core, allocating 100k per thread on a 16 core machine is only 1.56MB? And allocation should be pretty much instant. Though that would only give you 25 buffers per thread at 4096 per buffer.\r\n\r\n> Do you have links to a reference that talks about avoiding the first and last 32 bytes in a physical page?\r\n\r\nThe x86/x64 CPU<->RAM link reads and writes in atomic 64 byte chunks, which it puts in its 64 byte cache lines. It also does pre-fetching to get the next 64 byte chunk which is why arrays are super fast vs linked list (though the .net GC does it best to optimise for this) \r\n\r\nIf you want to deep dive, this is a good video: https://www.youtube.com/watch?v=L7zSU9HI-6I (1hr 51min)\r\n\r\nHowever; this 64 byte atomicity means if two threads on two different cores (or worse socket) operating on adjacent chunks of memory within the same cache line they have to invalidate each others cache and ping pong it back through RAM. As RAM is 200x slower than L1 cache and 20x slower than L2 cache this can cause CPU stalls; especially as the CPU runs at least x4 faster than even L1 cache.\r\n\r\nIf you are operating on a nice aligned 64 byte multiples (like 4096) then you are probably ok; however its still going to pre-fetch the start of another cache line; which it will then invalidate. So the 64 byte 'guard' buffer keeps down the hardware synchronisation the memory controller is doing. Also with aligned memory you are better adding a full 64 byte chunk at end.\r\n\r\nYou don't want to do this everywhere just where you may hit *nearby* write contention; which pooling will exacerbate; its often one of the main reasons why a simple implementation of an [embarrassingly parallel](https://en.wikipedia.org/wiki/Embarrassingly_parallel) algorithm won't scale with core count; even though there is no obvious code reason for it.\r\n\r\nLots of stuff on false sharing: https://en.wikipedia.org/wiki/False_sharing\r\n\r\nhttp://mechanical-sympathy.blogspot.co.uk/2011/07/false-sharing.html\r\nhttp://www.drdobbs.com/parallel/eliminate-false-sharing/217500206\r\nhttps://software.intel.com/en-us/articles/avoiding-and-identifying-false-sharing-among-threads\r\nhttp://www.codeproject.com/Articles/51553/Concurrency-Hazards-False-Sharing\r\n\r\n![image](https://cloud.githubusercontent.com/assets/1142958/9364562/f95a98aa-46a4-11e5-8524-85c709950b63.png)\r\n\r\nGenerally false sharing isn't something to worry about in day-to-day programming; but if Kestrel is doing 164k rps the performance is already way beyond that; that's a request served every 6 microseconds!\r\n\r\n"}}