{"action":"created","comment":{"url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/comments/29562896","id":29562896,"diff_hunk":"@@ -2,30 +2,104 @@\n #define BRICKS_MQ_INMEMORY_MQ_H\n \n // MMQ is an efficient in-memory FIFO buffer.\n+// One of the objectives of MMQ is to minimize the time for which the message pushing thread is blocked for.\n //\n-// Messages can be pushed into it via `PushMessage()`.\n+// Messages can be pushed into it via thread-safe methods `PushMessage()` or `EmplaceMessage()`.\n // The consumer is run in a separate thread, and is fed one message at a time via `OnMessage()`.\n-// The order of messages is preserved.\n //\n-// One of the objectives of MMQ is to minimize the time for which the message pushing thread is blocked for.\n+// The buffer size, i.e. the number of the messages MMQ can hold, is defined by\n+// the constructor argument `buffer_size`. For usability reasons the default\n+// value for it can be set via `DEFAULT_BUFFER_SIZE` template argument.\n //\n-// The default (and only so far) implementation discards older messages if the queue gets over capacity.\n-// TODO(dkorolev): This behavior should probably change.\n+// There are two possible strategies in case of buffer overflow (i.e. there is\n+// no free space to store message at the next call to `PushMessage()` or\n+// `EmplaceMessage()`):\n+//   1) Discard (drop) the message. In this case, the number of the messages dropped between\n+//      the subseqent calls of the consumer may be passed as a second argument of `OnMessage()`.\n+//   2) Block the pushing thread and wait for the next message to be consumed\n+//      and free the space in the buffer.\n+//      IMPORTANT NOTE: if there are several threads waiting to push the\n+//      message, MMQ DOES NOT guarantee that the messages will be added in the\n+//      order in which the functions were called. However, for any particular\n+//      thread MMQ DOES GUARANTEE the order of the messages for the subsequent\n+//      requests to push the message.\n+//  This behavior of MMQ can be controlled via `DROP_ON_OVERFLOW` template\n+//  argument.\n \n #include <condition_variable>\n #include <mutex>\n #include <string>\n #include <thread>\n #include <vector>\n+#include <iostream>\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+constexpr bool HasSimpleOnMessage(char) {\n+  return false;\n+}\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+constexpr auto HasSimpleOnMessage(int)\n+    -> decltype(std::declval<T_CONSUMER>().OnMessage(std::move(std::declval<T_MESSAGE>())), bool()) {\n+  return true;\n+}\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+constexpr bool HasExtendedOnMessage(char) {\n+  return false;\n+}\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+constexpr auto HasExtendedOnMessage(int)\n+    -> decltype(std::declval<T_CONSUMER>().OnMessage(std::move(std::declval<T_MESSAGE>()), 0u), bool()) {\n+  return true;\n+}\n+\n+template <typename T_CONSUMER, typename T_MESSAGE, bool HAS_SIMPLE_ON_MESSAGE>\n+struct ExportMessageImpl {};\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+struct ExportMessageImpl<T_CONSUMER, T_MESSAGE, true> {\n+  static void OnMessage(T_CONSUMER& consumer, T_MESSAGE&& message, size_t) {\n+    consumer.OnMessage(std::move(message));\n+  }\n+};\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+struct ExportMessageImpl<T_CONSUMER, T_MESSAGE, false> {\n+  static void OnMessage(T_CONSUMER& consumer, T_MESSAGE&& message, size_t dropped_count) {\n+    consumer.OnMessage(std::move(message), dropped_count);\n+  }\n+};\n+\n+template <typename T_CONSUMER, typename T_MESSAGE>\n+void ExportMessage(T_CONSUMER& consumer, T_MESSAGE&& message, size_t dropped_count) {\n+  static_assert(\n+      !(HasSimpleOnMessage<T_CONSUMER, T_MESSAGE>(0) && HasExtendedOnMessage<T_CONSUMER, T_MESSAGE>(0)),\n+      \"Two versions of `OnMessage()` are defined, choose one.\");\n+  ExportMessageImpl<T_CONSUMER, T_MESSAGE, HasSimpleOnMessage<T_CONSUMER, T_MESSAGE>(0)>::OnMessage(\n+      consumer, std::move(message), dropped_count);\n+}\n \n-template <typename CONSUMER, typename MESSAGE = std::string, size_t DEFAULT_BUFFER_SIZE = 1024>\n+template <typename CONSUMER,\n+          typename MESSAGE = std::string,\n+          size_t DEFAULT_BUFFER_SIZE = 1024,\n+          bool DROP_ON_OVERFLOW = true>\n class MMQ final {\n  public:\n   // Type of entries to store, defaults to `std::string`.\n   typedef MESSAGE T_MESSAGE;\n \n   // Type of the processor of the entries.\n-  // It should expose one method, void OnMessage(T_MESSAGE&, size_t number_of_dropped_messages_if_any);\n+  // It should expose one of the methods, accepting message by const or rvalue reference:\n+  //\n+  //   void OnMessage(T_MESSAGE&[&], size_t number_of_dropped_messages_if_any);","path":"mq/inmemory/mq.h","position":102,"original_position":102,"commit_id":"9dba811f6ed6d217597948db9e4f3bdeb9c92a01","original_commit_id":"9dba811f6ed6d217597948db9e4f3bdeb9c92a01","user":{"login":"dkorolev","id":2159447,"avatar_url":"https://avatars.githubusercontent.com/u/2159447?v=3","gravatar_id":"","url":"https://api.github.com/users/dkorolev","html_url":"https://github.com/dkorolev","followers_url":"https://api.github.com/users/dkorolev/followers","following_url":"https://api.github.com/users/dkorolev/following{/other_user}","gists_url":"https://api.github.com/users/dkorolev/gists{/gist_id}","starred_url":"https://api.github.com/users/dkorolev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkorolev/subscriptions","organizations_url":"https://api.github.com/users/dkorolev/orgs","repos_url":"https://api.github.com/users/dkorolev/repos","events_url":"https://api.github.com/users/dkorolev/events{/privacy}","received_events_url":"https://api.github.com/users/dkorolev/received_events","type":"User","site_admin":false},"body":"I know it looks redundant, but `[const] ` would also fit the prototype. Why not show the `T_MESSAGE&&` syntax, and on the next line put `(Alternatively, void OnMessage(const T_MESSAGE&, size_t)` ?","created_at":"2015-05-04T01:23:16Z","updated_at":"2015-05-04T01:23:16Z","html_url":"https://github.com/KnowSheet/Bricks/pull/155#discussion_r29562896","pull_request_url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155","_links":{"self":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/comments/29562896"},"html":{"href":"https://github.com/KnowSheet/Bricks/pull/155#discussion_r29562896"},"pull_request":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155"}}},"pull_request":{"url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155","id":34611373,"html_url":"https://github.com/KnowSheet/Bricks/pull/155","diff_url":"https://github.com/KnowSheet/Bricks/pull/155.diff","patch_url":"https://github.com/KnowSheet/Bricks/pull/155.patch","issue_url":"https://api.github.com/repos/KnowSheet/Bricks/issues/155","number":155,"state":"open","locked":false,"title":"MMQ with drop-or-wait policy on overflow + tests","user":{"login":"mzhurovich","id":11226961,"avatar_url":"https://avatars.githubusercontent.com/u/11226961?v=3","gravatar_id":"","url":"https://api.github.com/users/mzhurovich","html_url":"https://github.com/mzhurovich","followers_url":"https://api.github.com/users/mzhurovich/followers","following_url":"https://api.github.com/users/mzhurovich/following{/other_user}","gists_url":"https://api.github.com/users/mzhurovich/gists{/gist_id}","starred_url":"https://api.github.com/users/mzhurovich/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mzhurovich/subscriptions","organizations_url":"https://api.github.com/users/mzhurovich/orgs","repos_url":"https://api.github.com/users/mzhurovich/repos","events_url":"https://api.github.com/users/mzhurovich/events{/privacy}","received_events_url":"https://api.github.com/users/mzhurovich/received_events","type":"User","site_admin":false},"body":"@dkorolev Looks like I more-or-less finished it, PTAL\r\n\r\nCC @deathbaba: Dima said, it could be useful for you - we fixed a bug with dropping MMQ and added an option to make it non-dropping","created_at":"2015-05-04T01:10:19Z","updated_at":"2015-05-04T01:23:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"738581d616573e4394b2a67bf2b594132744a0a9","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155/commits","review_comments_url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155/comments","review_comment_url":"https://api.github.com/repos/KnowSheet/Bricks/pulls/comments{/number}","comments_url":"https://api.github.com/repos/KnowSheet/Bricks/issues/155/comments","statuses_url":"https://api.github.com/repos/KnowSheet/Bricks/statuses/9dba811f6ed6d217597948db9e4f3bdeb9c92a01","head":{"label":"mzhurovich:mmq","ref":"mmq","sha":"9dba811f6ed6d217597948db9e4f3bdeb9c92a01","user":{"login":"mzhurovich","id":11226961,"avatar_url":"https://avatars.githubusercontent.com/u/11226961?v=3","gravatar_id":"","url":"https://api.github.com/users/mzhurovich","html_url":"https://github.com/mzhurovich","followers_url":"https://api.github.com/users/mzhurovich/followers","following_url":"https://api.github.com/users/mzhurovich/following{/other_user}","gists_url":"https://api.github.com/users/mzhurovich/gists{/gist_id}","starred_url":"https://api.github.com/users/mzhurovich/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mzhurovich/subscriptions","organizations_url":"https://api.github.com/users/mzhurovich/orgs","repos_url":"https://api.github.com/users/mzhurovich/repos","events_url":"https://api.github.com/users/mzhurovich/events{/privacy}","received_events_url":"https://api.github.com/users/mzhurovich/received_events","type":"User","site_admin":false},"repo":{"id":31462826,"name":"Bricks","full_name":"mzhurovich/Bricks","owner":{"login":"mzhurovich","id":11226961,"avatar_url":"https://avatars.githubusercontent.com/u/11226961?v=3","gravatar_id":"","url":"https://api.github.com/users/mzhurovich","html_url":"https://github.com/mzhurovich","followers_url":"https://api.github.com/users/mzhurovich/followers","following_url":"https://api.github.com/users/mzhurovich/following{/other_user}","gists_url":"https://api.github.com/users/mzhurovich/gists{/gist_id}","starred_url":"https://api.github.com/users/mzhurovich/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mzhurovich/subscriptions","organizations_url":"https://api.github.com/users/mzhurovich/orgs","repos_url":"https://api.github.com/users/mzhurovich/repos","events_url":"https://api.github.com/users/mzhurovich/events{/privacy}","received_events_url":"https://api.github.com/users/mzhurovich/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mzhurovich/Bricks","description":"Our own core pieces to reuse.","fork":true,"url":"https://api.github.com/repos/mzhurovich/Bricks","forks_url":"https://api.github.com/repos/mzhurovich/Bricks/forks","keys_url":"https://api.github.com/repos/mzhurovich/Bricks/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mzhurovich/Bricks/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mzhurovich/Bricks/teams","hooks_url":"https://api.github.com/repos/mzhurovich/Bricks/hooks","issue_events_url":"https://api.github.com/repos/mzhurovich/Bricks/issues/events{/number}","events_url":"https://api.github.com/repos/mzhurovich/Bricks/events","assignees_url":"https://api.github.com/repos/mzhurovich/Bricks/assignees{/user}","branches_url":"https://api.github.com/repos/mzhurovich/Bricks/branches{/branch}","tags_url":"https://api.github.com/repos/mzhurovich/Bricks/tags","blobs_url":"https://api.github.com/repos/mzhurovich/Bricks/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mzhurovich/Bricks/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mzhurovich/Bricks/git/refs{/sha}","trees_url":"https://api.github.com/repos/mzhurovich/Bricks/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mzhurovich/Bricks/statuses/{sha}","languages_url":"https://api.github.com/repos/mzhurovich/Bricks/languages","stargazers_url":"https://api.github.com/repos/mzhurovich/Bricks/stargazers","contributors_url":"https://api.github.com/repos/mzhurovich/Bricks/contributors","subscribers_url":"https://api.github.com/repos/mzhurovich/Bricks/subscribers","subscription_url":"https://api.github.com/repos/mzhurovich/Bricks/subscription","commits_url":"https://api.github.com/repos/mzhurovich/Bricks/commits{/sha}","git_commits_url":"https://api.github.com/repos/mzhurovich/Bricks/git/commits{/sha}","comments_url":"https://api.github.com/repos/mzhurovich/Bricks/comments{/number}","issue_comment_url":"https://api.github.com/repos/mzhurovich/Bricks/issues/comments{/number}","contents_url":"https://api.github.com/repos/mzhurovich/Bricks/contents/{+path}","compare_url":"https://api.github.com/repos/mzhurovich/Bricks/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mzhurovich/Bricks/merges","archive_url":"https://api.github.com/repos/mzhurovich/Bricks/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mzhurovich/Bricks/downloads","issues_url":"https://api.github.com/repos/mzhurovich/Bricks/issues{/number}","pulls_url":"https://api.github.com/repos/mzhurovich/Bricks/pulls{/number}","milestones_url":"https://api.github.com/repos/mzhurovich/Bricks/milestones{/number}","notifications_url":"https://api.github.com/repos/mzhurovich/Bricks/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mzhurovich/Bricks/labels{/name}","releases_url":"https://api.github.com/repos/mzhurovich/Bricks/releases{/id}","created_at":"2015-02-28T13:03:55Z","updated_at":"2015-05-02T09:32:36Z","pushed_at":"2015-05-04T01:05:09Z","git_url":"git://github.com/mzhurovich/Bricks.git","ssh_url":"git@github.com:mzhurovich/Bricks.git","clone_url":"https://github.com/mzhurovich/Bricks.git","svn_url":"https://github.com/mzhurovich/Bricks","homepage":null,"size":7686,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"KnowSheet:master","ref":"master","sha":"eca471bb8b748a190ed4861789e0109c00bd5844","user":{"login":"KnowSheet","id":9897562,"avatar_url":"https://avatars.githubusercontent.com/u/9897562?v=3","gravatar_id":"","url":"https://api.github.com/users/KnowSheet","html_url":"https://github.com/KnowSheet","followers_url":"https://api.github.com/users/KnowSheet/followers","following_url":"https://api.github.com/users/KnowSheet/following{/other_user}","gists_url":"https://api.github.com/users/KnowSheet/gists{/gist_id}","starred_url":"https://api.github.com/users/KnowSheet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KnowSheet/subscriptions","organizations_url":"https://api.github.com/users/KnowSheet/orgs","repos_url":"https://api.github.com/users/KnowSheet/repos","events_url":"https://api.github.com/users/KnowSheet/events{/privacy}","received_events_url":"https://api.github.com/users/KnowSheet/received_events","type":"Organization","site_admin":false},"repo":{"id":27625425,"name":"Bricks","full_name":"KnowSheet/Bricks","owner":{"login":"KnowSheet","id":9897562,"avatar_url":"https://avatars.githubusercontent.com/u/9897562?v=3","gravatar_id":"","url":"https://api.github.com/users/KnowSheet","html_url":"https://github.com/KnowSheet","followers_url":"https://api.github.com/users/KnowSheet/followers","following_url":"https://api.github.com/users/KnowSheet/following{/other_user}","gists_url":"https://api.github.com/users/KnowSheet/gists{/gist_id}","starred_url":"https://api.github.com/users/KnowSheet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KnowSheet/subscriptions","organizations_url":"https://api.github.com/users/KnowSheet/orgs","repos_url":"https://api.github.com/users/KnowSheet/repos","events_url":"https://api.github.com/users/KnowSheet/events{/privacy}","received_events_url":"https://api.github.com/users/KnowSheet/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/KnowSheet/Bricks","description":"HTTP server and client, JSON and binary serialization, visualization and more. Pure header-only C++11.","fork":false,"url":"https://api.github.com/repos/KnowSheet/Bricks","forks_url":"https://api.github.com/repos/KnowSheet/Bricks/forks","keys_url":"https://api.github.com/repos/KnowSheet/Bricks/keys{/key_id}","collaborators_url":"https://api.github.com/repos/KnowSheet/Bricks/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/KnowSheet/Bricks/teams","hooks_url":"https://api.github.com/repos/KnowSheet/Bricks/hooks","issue_events_url":"https://api.github.com/repos/KnowSheet/Bricks/issues/events{/number}","events_url":"https://api.github.com/repos/KnowSheet/Bricks/events","assignees_url":"https://api.github.com/repos/KnowSheet/Bricks/assignees{/user}","branches_url":"https://api.github.com/repos/KnowSheet/Bricks/branches{/branch}","tags_url":"https://api.github.com/repos/KnowSheet/Bricks/tags","blobs_url":"https://api.github.com/repos/KnowSheet/Bricks/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/KnowSheet/Bricks/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/KnowSheet/Bricks/git/refs{/sha}","trees_url":"https://api.github.com/repos/KnowSheet/Bricks/git/trees{/sha}","statuses_url":"https://api.github.com/repos/KnowSheet/Bricks/statuses/{sha}","languages_url":"https://api.github.com/repos/KnowSheet/Bricks/languages","stargazers_url":"https://api.github.com/repos/KnowSheet/Bricks/stargazers","contributors_url":"https://api.github.com/repos/KnowSheet/Bricks/contributors","subscribers_url":"https://api.github.com/repos/KnowSheet/Bricks/subscribers","subscription_url":"https://api.github.com/repos/KnowSheet/Bricks/subscription","commits_url":"https://api.github.com/repos/KnowSheet/Bricks/commits{/sha}","git_commits_url":"https://api.github.com/repos/KnowSheet/Bricks/git/commits{/sha}","comments_url":"https://api.github.com/repos/KnowSheet/Bricks/comments{/number}","issue_comment_url":"https://api.github.com/repos/KnowSheet/Bricks/issues/comments{/number}","contents_url":"https://api.github.com/repos/KnowSheet/Bricks/contents/{+path}","compare_url":"https://api.github.com/repos/KnowSheet/Bricks/compare/{base}...{head}","merges_url":"https://api.github.com/repos/KnowSheet/Bricks/merges","archive_url":"https://api.github.com/repos/KnowSheet/Bricks/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/KnowSheet/Bricks/downloads","issues_url":"https://api.github.com/repos/KnowSheet/Bricks/issues{/number}","pulls_url":"https://api.github.com/repos/KnowSheet/Bricks/pulls{/number}","milestones_url":"https://api.github.com/repos/KnowSheet/Bricks/milestones{/number}","notifications_url":"https://api.github.com/repos/KnowSheet/Bricks/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/KnowSheet/Bricks/labels{/name}","releases_url":"https://api.github.com/repos/KnowSheet/Bricks/releases{/id}","created_at":"2014-12-06T06:52:23Z","updated_at":"2015-04-28T16:42:27Z","pushed_at":"2015-05-04T01:10:19Z","git_url":"git://github.com/KnowSheet/Bricks.git","ssh_url":"git@github.com:KnowSheet/Bricks.git","clone_url":"https://github.com/KnowSheet/Bricks.git","svn_url":"https://github.com/KnowSheet/Bricks","homepage":"","size":9145,"stargazers_count":9,"watchers_count":9,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":7,"mirror_url":null,"open_issues_count":30,"forks":7,"open_issues":30,"watchers":9,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155"},"html":{"href":"https://github.com/KnowSheet/Bricks/pull/155"},"issue":{"href":"https://api.github.com/repos/KnowSheet/Bricks/issues/155"},"comments":{"href":"https://api.github.com/repos/KnowSheet/Bricks/issues/155/comments"},"review_comments":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155/comments"},"review_comment":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/KnowSheet/Bricks/pulls/155/commits"},"statuses":{"href":"https://api.github.com/repos/KnowSheet/Bricks/statuses/9dba811f6ed6d217597948db9e4f3bdeb9c92a01"}}}}