{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-apex-core/issues/173","labels_url":"https://api.github.com/repos/apache/incubator-apex-core/issues/173/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-apex-core/issues/173/comments","events_url":"https://api.github.com/repos/apache/incubator-apex-core/issues/173/events","html_url":"https://github.com/apache/incubator-apex-core/pull/173","id":117493667,"number":173,"title":"APEX-254 & APEX-269","user":{"login":"vrozov","id":12200381,"avatar_url":"https://avatars.githubusercontent.com/u/12200381?v=3","gravatar_id":"","url":"https://api.github.com/users/vrozov","html_url":"https://github.com/vrozov","followers_url":"https://api.github.com/users/vrozov/followers","following_url":"https://api.github.com/users/vrozov/following{/other_user}","gists_url":"https://api.github.com/users/vrozov/gists{/gist_id}","starred_url":"https://api.github.com/users/vrozov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vrozov/subscriptions","organizations_url":"https://api.github.com/users/vrozov/orgs","repos_url":"https://api.github.com/users/vrozov/repos","events_url":"https://api.github.com/users/vrozov/events{/privacy}","received_events_url":"https://api.github.com/users/vrozov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2015-11-18T01:56:47Z","updated_at":"2015-11-25T23:07:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/incubator-apex-core/pulls/173","html_url":"https://github.com/apache/incubator-apex-core/pull/173","diff_url":"https://github.com/apache/incubator-apex-core/pull/173.diff","patch_url":"https://github.com/apache/incubator-apex-core/pull/173.patch"},"body":"Introduce Abstract and Forwarding Reservoir classes. Provide concrete implementation of AbstractReservoir based on JCTools SpscArrayQueue.\r\n\r\nSpscArrayQueueReservoir allows to process 25 million records per second for ports with CONTAINER_LOCAL locality.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/incubator-apex-core/issues/comments/159751791","html_url":"https://github.com/apache/incubator-apex-core/pull/173#issuecomment-159751791","issue_url":"https://api.github.com/repos/apache/incubator-apex-core/issues/173","id":159751791,"user":{"login":"243826","id":2702787,"avatar_url":"https://avatars.githubusercontent.com/u/2702787?v=3","gravatar_id":"","url":"https://api.github.com/users/243826","html_url":"https://github.com/243826","followers_url":"https://api.github.com/users/243826/followers","following_url":"https://api.github.com/users/243826/following{/other_user}","gists_url":"https://api.github.com/users/243826/gists{/gist_id}","starred_url":"https://api.github.com/users/243826/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/243826/subscriptions","organizations_url":"https://api.github.com/users/243826/orgs","repos_url":"https://api.github.com/users/243826/repos","events_url":"https://api.github.com/users/243826/events{/privacy}","received_events_url":"https://api.github.com/users/243826/received_events","type":"User","site_admin":false},"created_at":"2015-11-25T23:07:16Z","updated_at":"2015-11-25T23:07:16Z","body":"It's not a good idea. It's micro optimization (of not a critical path) at the cost of complicated implementation for developers and complicated use for the users.\n\nIn the benchmark that Vlad shared starting at 128k queue size we start seeing that the cost of receiving the event outweighs the cost of processing it - this is unsustainable practically unless we are doing next to none processing of the event. I even suspect that it's the same event that's being pushed as opposed to pushing a new event every time through SPSC (Vlad - do you have reuse_tuple or equivalent flag turned on?).  Practically a long before we reach that throughput with larger queue size - we breach through the barrier between the ContainerLocal and ThreadLocal. The timing is driven by the processing complexity of subscriber operator. At that point you may as well switch to ThreadLocal and enjoy much more and optimal throughput (the benchmark for ThreadLocal it also very academic).\n\n-\nChetan\n\n\n\nFrom: Thomas Weise <notifications@github.com<mailto:notifications@github.com>>\nReply-To: apache/incubator-apex-core <reply@reply.github.com<mailto:reply@reply.github.com>>\nDate: Wednesday, November 25, 2015 at 12:54 PM\nTo: apache/incubator-apex-core <incubator-apex-core@noreply.github.com<mailto:incubator-apex-core@noreply.github.com>>\nCc: Chetan Narsude <chetan@apache.org<mailto:chetan@apache.org>>\nSubject: Re: [incubator-apex-core] APEX-254 & APEX-269 (#173)\n\n\nVlad, this is great data. How about selecting the default reservoir implementation based on the queue size? ArrayBlockingQueueReservoir for small queue size, spsc for larger size? And leave the final say about the implementation to the user?\n\n-\nReply to this email directly or view it on GitHub<https://github.com/apache/incubator-apex-core/pull/173#issuecomment-159729354>.\n"}}