{"action":"created","comment":{"url":"https://api.github.com/repos/reverbrain/eblob/pulls/comments/25888267","id":25888267,"diff_hunk":"@@ -52,6 +53,29 @@ static void eblob_extract_id(const struct eblob_id &e, struct eblob_key &id)\n \teblob_extract_arr(e.id, id.id, &len);\n }\n \n+/**\n+ * __eblob_read_ll() - interruption-safe wrapper for pread(2)\n+ */\n+static int __eblob_read_ll(int fd, char *data, size_t size, off_t offset)","path":"bindings/python/eblob_python.cpp","position":15,"original_position":15,"commit_id":"4c62ffe71030dc9c3b930db421ebfacb777ccbbb","original_commit_id":"4c62ffe71030dc9c3b930db421ebfacb777ccbbb","user":{"login":"bioothod","id":1847575,"avatar_url":"https://avatars.githubusercontent.com/u/1847575?v=3","gravatar_id":"","url":"https://api.github.com/users/bioothod","html_url":"https://github.com/bioothod","followers_url":"https://api.github.com/users/bioothod/followers","following_url":"https://api.github.com/users/bioothod/following{/other_user}","gists_url":"https://api.github.com/users/bioothod/gists{/gist_id}","starred_url":"https://api.github.com/users/bioothod/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bioothod/subscriptions","organizations_url":"https://api.github.com/users/bioothod/orgs","repos_url":"https://api.github.com/users/bioothod/repos","events_url":"https://api.github.com/users/bioothod/events{/privacy}","received_events_url":"https://api.github.com/users/bioothod/received_events","type":"User","site_admin":false},"body":"It is not right now, so was the question.","created_at":"2015-03-05T18:30:40Z","updated_at":"2015-03-05T18:30:40Z","html_url":"https://github.com/reverbrain/eblob/pull/104#discussion_r25888267","pull_request_url":"https://api.github.com/repos/reverbrain/eblob/pulls/104","_links":{"self":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/comments/25888267"},"html":{"href":"https://github.com/reverbrain/eblob/pull/104#discussion_r25888267"},"pull_request":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/104"}}},"pull_request":{"url":"https://api.github.com/repos/reverbrain/eblob/pulls/104","id":30479682,"html_url":"https://github.com/reverbrain/eblob/pull/104","diff_url":"https://github.com/reverbrain/eblob/pull/104.diff","patch_url":"https://github.com/reverbrain/eblob/pull/104.patch","issue_url":"https://api.github.com/repos/reverbrain/eblob/issues/104","number":104,"state":"open","locked":false,"title":"mmap elimination","user":{"login":"abudnik","id":1283969,"avatar_url":"https://avatars.githubusercontent.com/u/1283969?v=3","gravatar_id":"","url":"https://api.github.com/users/abudnik","html_url":"https://github.com/abudnik","followers_url":"https://api.github.com/users/abudnik/followers","following_url":"https://api.github.com/users/abudnik/following{/other_user}","gists_url":"https://api.github.com/users/abudnik/gists{/gist_id}","starred_url":"https://api.github.com/users/abudnik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abudnik/subscriptions","organizations_url":"https://api.github.com/users/abudnik/orgs","repos_url":"https://api.github.com/users/abudnik/repos","events_url":"https://api.github.com/users/abudnik/events{/privacy}","received_events_url":"https://api.github.com/users/abudnik/received_events","type":"User","site_admin":false},"body":"1) Problem description\r\nIf error happens at block i/o device level, then access to memory mapped file, which is stored in this device, may lead to program termination with SIGBUS signal.\r\nWhen elliptics uses multiple backends, then problems of some backend may lead to malfunction of all backends. Elliptics (dnet_ioserv) nodes used to work with one backend only (some time ago), therefore increasing network traffic (route list updates), but isolated problems of a particular backend.\r\n\r\n2) Problem solution\r\nThis PR provides eblob version without usage of memory-mapped files. Now elliptics/eblob use syscalls for read/write operations (pread, pwrite), which returns error code in case of any i/o errors.\r\n\r\n3) Estimation of i/o operation cost (pread vs. mmap)\r\nUsing pread/pwrite for i/o operations leads to additional overhead and resources consumption comparing to using of memory-mapped files. Every i/o operation consists of following steps:\r\nsystem call, data copying between user space buffers and kernel space, memory allocacation and dealocation for data buffers.\r\n\r\n4) Affected operations after elimination of mmap\r\n* Index element lookup in index file => all operation of read and write to eblob backend\r\n* Sorting of index files\r\n* Record iteration\r\n* Checksumming\r\n\r\n5) Performance comparison on synthetic load tests (pread vs. mmap)\r\nWe compared performance of pread vs. mmap version of eblob on following load tests:\r\n* read at 10k rps: records 0.5-2.5 Mb\r\n* read at 12k rps: records 100 bytes - 10 Kb\r\n* write from 4 writing nodes (elliptics client): records 100-500 bytes\r\n* write from 1 writing node: records 100-500 bytes\r\n* iterate all keys\r\n\r\nAll read and write load tests where aimed at single elliptics node with 16 backends.\r\nWrite tests lasts short time (few minutes) and long time (30 minutes) at high rps.\r\n\r\nThose tests (accessing small keys) has shown some performance divergence between mmap and pread versions.\r\nTests with heavy-weight keys didn't lead to any performances divergence.\r\nAnalyzing performance tests leads to next conclusion: in some tests response times increased a little bit, in other tests response times even decreased a little bit; if average and median rps differs, then it differs at a gap of 4-5 % in both sides (faster or slower).\r\nAlso pread version did not lead to changes in system load (cpu usage, context switches, interrupts, memory consumption and iowaits).","created_at":"2015-03-04T16:22:25Z","updated_at":"2015-03-05T18:30:40Z","closed_at":null,"merged_at":null,"merge_commit_sha":"123e309e6fcf402946ee11de0f153e9abe4f2602","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/reverbrain/eblob/pulls/104/commits","review_comments_url":"https://api.github.com/repos/reverbrain/eblob/pulls/104/comments","review_comment_url":"https://api.github.com/repos/reverbrain/eblob/pulls/comments{/number}","comments_url":"https://api.github.com/repos/reverbrain/eblob/issues/104/comments","statuses_url":"https://api.github.com/repos/reverbrain/eblob/statuses/4c62ffe71030dc9c3b930db421ebfacb777ccbbb","head":{"label":"abudnik:ELL-385","ref":"ELL-385","sha":"4c62ffe71030dc9c3b930db421ebfacb777ccbbb","user":{"login":"abudnik","id":1283969,"avatar_url":"https://avatars.githubusercontent.com/u/1283969?v=3","gravatar_id":"","url":"https://api.github.com/users/abudnik","html_url":"https://github.com/abudnik","followers_url":"https://api.github.com/users/abudnik/followers","following_url":"https://api.github.com/users/abudnik/following{/other_user}","gists_url":"https://api.github.com/users/abudnik/gists{/gist_id}","starred_url":"https://api.github.com/users/abudnik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abudnik/subscriptions","organizations_url":"https://api.github.com/users/abudnik/orgs","repos_url":"https://api.github.com/users/abudnik/repos","events_url":"https://api.github.com/users/abudnik/events{/privacy}","received_events_url":"https://api.github.com/users/abudnik/received_events","type":"User","site_admin":false},"repo":{"id":27128618,"name":"eblob","full_name":"abudnik/eblob","owner":{"login":"abudnik","id":1283969,"avatar_url":"https://avatars.githubusercontent.com/u/1283969?v=3","gravatar_id":"","url":"https://api.github.com/users/abudnik","html_url":"https://github.com/abudnik","followers_url":"https://api.github.com/users/abudnik/followers","following_url":"https://api.github.com/users/abudnik/following{/other_user}","gists_url":"https://api.github.com/users/abudnik/gists{/gist_id}","starred_url":"https://api.github.com/users/abudnik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abudnik/subscriptions","organizations_url":"https://api.github.com/users/abudnik/orgs","repos_url":"https://api.github.com/users/abudnik/repos","events_url":"https://api.github.com/users/abudnik/events{/privacy}","received_events_url":"https://api.github.com/users/abudnik/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/abudnik/eblob","description":"Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics","fork":true,"url":"https://api.github.com/repos/abudnik/eblob","forks_url":"https://api.github.com/repos/abudnik/eblob/forks","keys_url":"https://api.github.com/repos/abudnik/eblob/keys{/key_id}","collaborators_url":"https://api.github.com/repos/abudnik/eblob/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/abudnik/eblob/teams","hooks_url":"https://api.github.com/repos/abudnik/eblob/hooks","issue_events_url":"https://api.github.com/repos/abudnik/eblob/issues/events{/number}","events_url":"https://api.github.com/repos/abudnik/eblob/events","assignees_url":"https://api.github.com/repos/abudnik/eblob/assignees{/user}","branches_url":"https://api.github.com/repos/abudnik/eblob/branches{/branch}","tags_url":"https://api.github.com/repos/abudnik/eblob/tags","blobs_url":"https://api.github.com/repos/abudnik/eblob/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/abudnik/eblob/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/abudnik/eblob/git/refs{/sha}","trees_url":"https://api.github.com/repos/abudnik/eblob/git/trees{/sha}","statuses_url":"https://api.github.com/repos/abudnik/eblob/statuses/{sha}","languages_url":"https://api.github.com/repos/abudnik/eblob/languages","stargazers_url":"https://api.github.com/repos/abudnik/eblob/stargazers","contributors_url":"https://api.github.com/repos/abudnik/eblob/contributors","subscribers_url":"https://api.github.com/repos/abudnik/eblob/subscribers","subscription_url":"https://api.github.com/repos/abudnik/eblob/subscription","commits_url":"https://api.github.com/repos/abudnik/eblob/commits{/sha}","git_commits_url":"https://api.github.com/repos/abudnik/eblob/git/commits{/sha}","comments_url":"https://api.github.com/repos/abudnik/eblob/comments{/number}","issue_comment_url":"https://api.github.com/repos/abudnik/eblob/issues/comments{/number}","contents_url":"https://api.github.com/repos/abudnik/eblob/contents/{+path}","compare_url":"https://api.github.com/repos/abudnik/eblob/compare/{base}...{head}","merges_url":"https://api.github.com/repos/abudnik/eblob/merges","archive_url":"https://api.github.com/repos/abudnik/eblob/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/abudnik/eblob/downloads","issues_url":"https://api.github.com/repos/abudnik/eblob/issues{/number}","pulls_url":"https://api.github.com/repos/abudnik/eblob/pulls{/number}","milestones_url":"https://api.github.com/repos/abudnik/eblob/milestones{/number}","notifications_url":"https://api.github.com/repos/abudnik/eblob/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/abudnik/eblob/labels{/name}","releases_url":"https://api.github.com/repos/abudnik/eblob/releases{/id}","created_at":"2014-11-25T14:08:38Z","updated_at":"2015-02-04T11:36:20Z","pushed_at":"2015-03-04T16:17:13Z","git_url":"git://github.com/abudnik/eblob.git","ssh_url":"git@github.com:abudnik/eblob.git","clone_url":"https://github.com/abudnik/eblob.git","svn_url":"https://github.com/abudnik/eblob","homepage":null,"size":4632,"stargazers_count":0,"watchers_count":0,"language":"C","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"reverbrain:master","ref":"master","sha":"24464996ee316678821be26c22ea0ed8bfa990d9","user":{"login":"reverbrain","id":1616511,"avatar_url":"https://avatars.githubusercontent.com/u/1616511?v=3","gravatar_id":"","url":"https://api.github.com/users/reverbrain","html_url":"https://github.com/reverbrain","followers_url":"https://api.github.com/users/reverbrain/followers","following_url":"https://api.github.com/users/reverbrain/following{/other_user}","gists_url":"https://api.github.com/users/reverbrain/gists{/gist_id}","starred_url":"https://api.github.com/users/reverbrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reverbrain/subscriptions","organizations_url":"https://api.github.com/users/reverbrain/orgs","repos_url":"https://api.github.com/users/reverbrain/repos","events_url":"https://api.github.com/users/reverbrain/events{/privacy}","received_events_url":"https://api.github.com/users/reverbrain/received_events","type":"Organization","site_admin":false},"repo":{"id":4712280,"name":"eblob","full_name":"reverbrain/eblob","owner":{"login":"reverbrain","id":1616511,"avatar_url":"https://avatars.githubusercontent.com/u/1616511?v=3","gravatar_id":"","url":"https://api.github.com/users/reverbrain","html_url":"https://github.com/reverbrain","followers_url":"https://api.github.com/users/reverbrain/followers","following_url":"https://api.github.com/users/reverbrain/following{/other_user}","gists_url":"https://api.github.com/users/reverbrain/gists{/gist_id}","starred_url":"https://api.github.com/users/reverbrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reverbrain/subscriptions","organizations_url":"https://api.github.com/users/reverbrain/orgs","repos_url":"https://api.github.com/users/reverbrain/repos","events_url":"https://api.github.com/users/reverbrain/events{/privacy}","received_events_url":"https://api.github.com/users/reverbrain/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/reverbrain/eblob","description":"Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics","fork":false,"url":"https://api.github.com/repos/reverbrain/eblob","forks_url":"https://api.github.com/repos/reverbrain/eblob/forks","keys_url":"https://api.github.com/repos/reverbrain/eblob/keys{/key_id}","collaborators_url":"https://api.github.com/repos/reverbrain/eblob/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/reverbrain/eblob/teams","hooks_url":"https://api.github.com/repos/reverbrain/eblob/hooks","issue_events_url":"https://api.github.com/repos/reverbrain/eblob/issues/events{/number}","events_url":"https://api.github.com/repos/reverbrain/eblob/events","assignees_url":"https://api.github.com/repos/reverbrain/eblob/assignees{/user}","branches_url":"https://api.github.com/repos/reverbrain/eblob/branches{/branch}","tags_url":"https://api.github.com/repos/reverbrain/eblob/tags","blobs_url":"https://api.github.com/repos/reverbrain/eblob/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/reverbrain/eblob/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/reverbrain/eblob/git/refs{/sha}","trees_url":"https://api.github.com/repos/reverbrain/eblob/git/trees{/sha}","statuses_url":"https://api.github.com/repos/reverbrain/eblob/statuses/{sha}","languages_url":"https://api.github.com/repos/reverbrain/eblob/languages","stargazers_url":"https://api.github.com/repos/reverbrain/eblob/stargazers","contributors_url":"https://api.github.com/repos/reverbrain/eblob/contributors","subscribers_url":"https://api.github.com/repos/reverbrain/eblob/subscribers","subscription_url":"https://api.github.com/repos/reverbrain/eblob/subscription","commits_url":"https://api.github.com/repos/reverbrain/eblob/commits{/sha}","git_commits_url":"https://api.github.com/repos/reverbrain/eblob/git/commits{/sha}","comments_url":"https://api.github.com/repos/reverbrain/eblob/comments{/number}","issue_comment_url":"https://api.github.com/repos/reverbrain/eblob/issues/comments{/number}","contents_url":"https://api.github.com/repos/reverbrain/eblob/contents/{+path}","compare_url":"https://api.github.com/repos/reverbrain/eblob/compare/{base}...{head}","merges_url":"https://api.github.com/repos/reverbrain/eblob/merges","archive_url":"https://api.github.com/repos/reverbrain/eblob/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/reverbrain/eblob/downloads","issues_url":"https://api.github.com/repos/reverbrain/eblob/issues{/number}","pulls_url":"https://api.github.com/repos/reverbrain/eblob/pulls{/number}","milestones_url":"https://api.github.com/repos/reverbrain/eblob/milestones{/number}","notifications_url":"https://api.github.com/repos/reverbrain/eblob/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/reverbrain/eblob/labels{/name}","releases_url":"https://api.github.com/repos/reverbrain/eblob/releases{/id}","created_at":"2012-06-19T10:29:26Z","updated_at":"2015-02-04T11:23:10Z","pushed_at":"2015-02-04T11:23:09Z","git_url":"git://github.com/reverbrain/eblob.git","ssh_url":"git@github.com:reverbrain/eblob.git","clone_url":"https://github.com/reverbrain/eblob.git","svn_url":"https://github.com/reverbrain/eblob","homepage":null,"size":5959,"stargazers_count":66,"watchers_count":66,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":18,"mirror_url":null,"open_issues_count":1,"forks":18,"open_issues":1,"watchers":66,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/104"},"html":{"href":"https://github.com/reverbrain/eblob/pull/104"},"issue":{"href":"https://api.github.com/repos/reverbrain/eblob/issues/104"},"comments":{"href":"https://api.github.com/repos/reverbrain/eblob/issues/104/comments"},"review_comments":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/104/comments"},"review_comment":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/reverbrain/eblob/pulls/104/commits"},"statuses":{"href":"https://api.github.com/repos/reverbrain/eblob/statuses/4c62ffe71030dc9c3b930db421ebfacb777ccbbb"}}}}