{"action":"created","comment":{"url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/comments/23700114","id":23700114,"diff_hunk":"@@ -0,0 +1,124 @@\n+/**********************/\n+/*    Redis Plugin    */\n+/**********************/\n+\n+This ECL plugin library is for the persistent key/value store 'redis' (http://redis.io). Utilising the C API\n+'hiredis' (http://github.com/redis/hirediscontains) and the event manager library libev\n+(http://software.schmorp.de/pkg/libev.html). Dependency notes at bottom.\n+\n+It contains three modules - 'sync', 'async', and 'locking'. Each one containing the core GET/SET variations\n+(by ECL type) differing at their base communication layer with the redis servers. 'async' and 'locking' both\n+use the asynchronous hiredis API, and 'sync' the synchronous API. Due to the nature of ECL as a declarative\n+and non-functional language, no async activity breaches directly into the ECL layer in a usable manner, i.e.\n+callbacks can not be used by the ECL activities EVENT and/OR WAIT. This being said, due to the current\n+constraints the ECL plugin infrastructure, some async logic does actually surface into the ECL layer but is\n+not directly accessible by any ECL, other than via these plugin functions. Such logic only concerns the\n+pub/sub nature of redis based around a locking mechanism on keys.\n+\n+A full list of redis commands can be found at http://redis.io/commands of which the following are directly\n+wrapped in the sync module only:\n+\n+BOOLEAN Exists(CONST VARSTRING options, CONST VARSTRING key, UNSIGNED database = 0)\n+FlushDB(CONST VARSTRING options, UNSIGNED database = 0)\n+Del(CONST VARSTRING options, CONST VARSTRING key, UNSIGNED database = 0)\n+Persist(CONST VARSTRING options, CONST VARSTRING key, UNSIGNED database = 0)\n+Expire(CONST VARSTRING options, CONST VARSTRING key, UNSIGNED4 expire, UNSIGNED database = 0)\n+INTEGER DBSize(CONST VARSTRING options, UNSIGNED database = 0)\n+\n+The redis command 'INFO ALL' is called once per instance of server:port combination and is written to the log\n+file (sync only). Only sync connections to a single redis server:port are cached internally, changes in db are\n+handled internally by the redis command 'SELECT DB'.\n+\n+The core SET functions have the prototype pattern (e.g.)\n+SetString(CONST VARSTRING options, CONST VARSTRING key, CONST STRING value, UNSIGNED database = 0,  UNSIGNED4 expire = 0 /*days*/)\n+The core GET functions have the prototype pattern (e.g.)\n+STRING GetString(CONST VARSTRING options, CONST VARSTRING key, UNSIGNED database = 0)\n+'options' is the redis server ip:port in the format '--SERVER=<ip>:<port>'. This defaults to localhost:6379 if\n+omitted.\n+\n+This pattern differs in the 'locking' module taking the following forms:\n+\n+STRING SetString(UNSIGNED8 keyPtr, CONST STRING value, UNSIGNED4 expire = 0)\n+STRING GetString(UNSIGNED8 keyPtr)\n+\n+Note that SET also returns a value. This is the vaue just set and not a wrapper for the redis command\n+'GETSET'. See the section on locks.\n+\n+Within the sync and async modules, SETs overwrite their key values. The respective GETs (other than for\n+STRINGs) will fail upon reuqest for a non-existent key. If you're not sure use EXISTS first (certain\n+precaution has to be taken when using multiple plugin calls within ECL, regarding their actual execution\n+timing. Remember ECL is declarative and may decide to hoist your calls.) GetString will return an empty\n+STRING upon a cache miss.\n+\n+/**********************/\n+/*     Locking        */\n+/**********************/\n+The locking module was designed with a particular use in mind. This concerns the scenario where multiple\n+ECL clients/threads wish to read the same key, at the same time, from the redis server and encounter an\n+non-existent key (cache miss). It is thus desirable to be able to then SET this missing key however, only\n+once and by one client/thread. This is such as to prevent redundant evaluation/retrieval of the new value\n+within this race condition. The core of this design is to have one instant (the first/race-winner), 'lock'\n+the key and then evaluate/retrieve the new value. The others clients/threads (race-losers) subscribe to\n+the locked key and wait for the winner to set the new value and publish the fact that it has done so.\n+This is done using the redis pub/sub paradigm. Note - in this implementation all publishes contain as\n+their message the very value that was just set, thus not requiring the subscriber to send subsequent GET\n+(this is not just an efficiency concern).\n+\n+A note about the redis pub/sub mechanism is that it is completely independent from the keyspace and database\n+index. Subscribers do so to a 'channel' (any given char string) only and not the particular key in question.\n+Any publication on this channel is heard by all current subscribers regardless of database index. This being\n+said, this plugin implementation does not allow for direct channel selection, instead it IS in fact dependent\n+on the keyspace and database. This is done so in a one-to-one and unique fashion such that a given key on a\n+given server:port and database creates a unique channel that is independent of plugin call and client/thread.\n+\n+Since redis does not implicitly support locks, it is this 'channel', itself, that represents the notion of a\n+lock within this plugin library. These channels/locks are a particular string prefixed by the string\n+'redis_ecl_lock'. When locked the key's value contains this channel/lock string. This is so lock acknowledging\n+GETs can subscribe to the key via the correct channel, being the  locked keys vaue itself.","path":"plugins/redis/README","position":77,"original_position":77,"commit_id":"c0ed06144d14688aad959661194bc5c237a506a6","original_commit_id":"c0ed06144d14688aad959661194bc5c237a506a6","user":{"login":"ghalliday","id":902490,"avatar_url":"https://avatars.githubusercontent.com/u/902490?v=3","gravatar_id":"","url":"https://api.github.com/users/ghalliday","html_url":"https://github.com/ghalliday","followers_url":"https://api.github.com/users/ghalliday/followers","following_url":"https://api.github.com/users/ghalliday/following{/other_user}","gists_url":"https://api.github.com/users/ghalliday/gists{/gist_id}","starred_url":"https://api.github.com/users/ghalliday/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ghalliday/subscriptions","organizations_url":"https://api.github.com/users/ghalliday/orgs","repos_url":"https://api.github.com/users/ghalliday/repos","events_url":"https://api.github.com/users/ghalliday/events{/privacy}","received_events_url":"https://api.github.com/users/ghalliday/received_events","type":"User","site_admin":false},"body":"value.  Double space.","created_at":"2015-01-28T16:37:52Z","updated_at":"2015-01-28T16:37:52Z","html_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900#discussion_r23700114","pull_request_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900","_links":{"self":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/comments/23700114"},"html":{"href":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900#discussion_r23700114"},"pull_request":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900"}}},"pull_request":{"url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900","id":28106225,"html_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900","diff_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900.diff","patch_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900.patch","issue_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/6900","number":6900,"state":"open","locked":false,"title":"HPCC-12395 Redis plugin","user":{"login":"jamienoss","id":5013975,"avatar_url":"https://avatars.githubusercontent.com/u/5013975?v=3","gravatar_id":"","url":"https://api.github.com/users/jamienoss","html_url":"https://github.com/jamienoss","followers_url":"https://api.github.com/users/jamienoss/followers","following_url":"https://api.github.com/users/jamienoss/following{/other_user}","gists_url":"https://api.github.com/users/jamienoss/gists{/gist_id}","starred_url":"https://api.github.com/users/jamienoss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamienoss/subscriptions","organizations_url":"https://api.github.com/users/jamienoss/orgs","repos_url":"https://api.github.com/users/jamienoss/repos","events_url":"https://api.github.com/users/jamienoss/events{/privacy}","received_events_url":"https://api.github.com/users/jamienoss/received_events","type":"User","site_admin":false},"body":"See README.\r\n\r\nSigned-off-by: james <james.noss@lexisnexis.com>","created_at":"2015-01-27T12:58:17Z","updated_at":"2015-01-28T16:37:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"09920d9aaea2d94f627f96dbb7d1e2160ad31652","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900/commits","review_comments_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900/comments","review_comment_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/comments/{number}","comments_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/6900/comments","statuses_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/statuses/c0ed06144d14688aad959661194bc5c237a506a6","head":{"label":"jamienoss:issue12395-support-redis-5.2.0rc1","ref":"issue12395-support-redis-5.2.0rc1","sha":"c0ed06144d14688aad959661194bc5c237a506a6","user":{"login":"jamienoss","id":5013975,"avatar_url":"https://avatars.githubusercontent.com/u/5013975?v=3","gravatar_id":"","url":"https://api.github.com/users/jamienoss","html_url":"https://github.com/jamienoss","followers_url":"https://api.github.com/users/jamienoss/followers","following_url":"https://api.github.com/users/jamienoss/following{/other_user}","gists_url":"https://api.github.com/users/jamienoss/gists{/gist_id}","starred_url":"https://api.github.com/users/jamienoss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamienoss/subscriptions","organizations_url":"https://api.github.com/users/jamienoss/orgs","repos_url":"https://api.github.com/users/jamienoss/repos","events_url":"https://api.github.com/users/jamienoss/events{/privacy}","received_events_url":"https://api.github.com/users/jamienoss/received_events","type":"User","site_admin":false},"repo":{"id":12548948,"name":"HPCC-Platform","full_name":"jamienoss/HPCC-Platform","owner":{"login":"jamienoss","id":5013975,"avatar_url":"https://avatars.githubusercontent.com/u/5013975?v=3","gravatar_id":"","url":"https://api.github.com/users/jamienoss","html_url":"https://github.com/jamienoss","followers_url":"https://api.github.com/users/jamienoss/followers","following_url":"https://api.github.com/users/jamienoss/following{/other_user}","gists_url":"https://api.github.com/users/jamienoss/gists{/gist_id}","starred_url":"https://api.github.com/users/jamienoss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamienoss/subscriptions","organizations_url":"https://api.github.com/users/jamienoss/orgs","repos_url":"https://api.github.com/users/jamienoss/repos","events_url":"https://api.github.com/users/jamienoss/events{/privacy}","received_events_url":"https://api.github.com/users/jamienoss/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jamienoss/HPCC-Platform","description":"IMPORTANT - Issue tracking has been migrated to JIRA - use your github ID and password reset mechanism to log in","fork":true,"url":"https://api.github.com/repos/jamienoss/HPCC-Platform","forks_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/forks","keys_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/teams","hooks_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/hooks","issue_events_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/issues/events{/number}","events_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/events","assignees_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/assignees{/user}","branches_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/branches{/branch}","tags_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/tags","blobs_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/git/refs{/sha}","trees_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/statuses/{sha}","languages_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/languages","stargazers_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/stargazers","contributors_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/contributors","subscribers_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/subscribers","subscription_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/subscription","commits_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/commits{/sha}","git_commits_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/git/commits{/sha}","comments_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/comments{/number}","issue_comment_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/issues/comments/{number}","contents_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/contents/{+path}","compare_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/merges","archive_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/downloads","issues_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/issues{/number}","pulls_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/pulls{/number}","milestones_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/milestones{/number}","notifications_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/labels{/name}","releases_url":"https://api.github.com/repos/jamienoss/HPCC-Platform/releases{/id}","created_at":"2013-09-02T20:57:07Z","updated_at":"2014-12-20T18:48:40Z","pushed_at":"2015-01-28T11:23:35Z","git_url":"git://github.com/jamienoss/HPCC-Platform.git","ssh_url":"git@github.com:jamienoss/HPCC-Platform.git","clone_url":"https://github.com/jamienoss/HPCC-Platform.git","svn_url":"https://github.com/jamienoss/HPCC-Platform","homepage":"track.hpccsystems.com","size":133689,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"hpcc-systems:candidate-5.2.0","ref":"candidate-5.2.0","sha":"aa983175dde93d658bfe159c352c98163b5d2448","user":{"login":"hpcc-systems","id":900867,"avatar_url":"https://avatars.githubusercontent.com/u/900867?v=3","gravatar_id":"","url":"https://api.github.com/users/hpcc-systems","html_url":"https://github.com/hpcc-systems","followers_url":"https://api.github.com/users/hpcc-systems/followers","following_url":"https://api.github.com/users/hpcc-systems/following{/other_user}","gists_url":"https://api.github.com/users/hpcc-systems/gists{/gist_id}","starred_url":"https://api.github.com/users/hpcc-systems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hpcc-systems/subscriptions","organizations_url":"https://api.github.com/users/hpcc-systems/orgs","repos_url":"https://api.github.com/users/hpcc-systems/repos","events_url":"https://api.github.com/users/hpcc-systems/events{/privacy}","received_events_url":"https://api.github.com/users/hpcc-systems/received_events","type":"Organization","site_admin":false},"repo":{"id":2030681,"name":"HPCC-Platform","full_name":"hpcc-systems/HPCC-Platform","owner":{"login":"hpcc-systems","id":900867,"avatar_url":"https://avatars.githubusercontent.com/u/900867?v=3","gravatar_id":"","url":"https://api.github.com/users/hpcc-systems","html_url":"https://github.com/hpcc-systems","followers_url":"https://api.github.com/users/hpcc-systems/followers","following_url":"https://api.github.com/users/hpcc-systems/following{/other_user}","gists_url":"https://api.github.com/users/hpcc-systems/gists{/gist_id}","starred_url":"https://api.github.com/users/hpcc-systems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hpcc-systems/subscriptions","organizations_url":"https://api.github.com/users/hpcc-systems/orgs","repos_url":"https://api.github.com/users/hpcc-systems/repos","events_url":"https://api.github.com/users/hpcc-systems/events{/privacy}","received_events_url":"https://api.github.com/users/hpcc-systems/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/hpcc-systems/HPCC-Platform","description":"IMPORTANT - Issue tracking has been migrated to JIRA - use your github ID and password reset mechanism to log in","fork":false,"url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform","forks_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/forks","keys_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/teams","hooks_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/hooks","issue_events_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/events{/number}","events_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/events","assignees_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/assignees{/user}","branches_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/branches{/branch}","tags_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/tags","blobs_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/git/refs{/sha}","trees_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/statuses/{sha}","languages_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/languages","stargazers_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/stargazers","contributors_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/contributors","subscribers_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/subscribers","subscription_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/subscription","commits_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/commits{/sha}","git_commits_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/git/commits{/sha}","comments_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/comments{/number}","issue_comment_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/comments/{number}","contents_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/contents/{+path}","compare_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/merges","archive_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/downloads","issues_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues{/number}","pulls_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls{/number}","milestones_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/milestones{/number}","notifications_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/labels{/name}","releases_url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/releases{/id}","created_at":"2011-07-11T14:29:51Z","updated_at":"2015-01-26T14:09:26Z","pushed_at":"2015-01-28T16:32:27Z","git_url":"git://github.com/hpcc-systems/HPCC-Platform.git","ssh_url":"git@github.com:hpcc-systems/HPCC-Platform.git","clone_url":"https://github.com/hpcc-systems/HPCC-Platform.git","svn_url":"https://github.com/hpcc-systems/HPCC-Platform","homepage":"track.hpccsystems.com","size":263751,"stargazers_count":241,"watchers_count":241,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":95,"mirror_url":null,"open_issues_count":419,"forks":95,"open_issues":419,"watchers":241,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900"},"html":{"href":"https://github.com/hpcc-systems/HPCC-Platform/pull/6900"},"issue":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/6900"},"comments":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/6900/comments"},"review_comments":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900/comments"},"review_comment":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/comments/{number}"},"commits":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/pulls/6900/commits"},"statuses":{"href":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/statuses/c0ed06144d14688aad959661194bc5c237a506a6"}}}}