{"action":"created","issue":{"url":"https://api.github.com/repos/SyncFree/antidote/issues/139","labels_url":"https://api.github.com/repos/SyncFree/antidote/issues/139/labels{/name}","comments_url":"https://api.github.com/repos/SyncFree/antidote/issues/139/comments","events_url":"https://api.github.com/repos/SyncFree/antidote/issues/139/events","html_url":"https://github.com/SyncFree/antidote/pull/139","id":91292125,"number":139,"title":"Materialiser fixes/optimisations + read/write path optimisations + merge with evaluation branch","user":{"login":"tcrain","id":5465025,"avatar_url":"https://avatars.githubusercontent.com/u/5465025?v=3","gravatar_id":"","url":"https://api.github.com/users/tcrain","html_url":"https://github.com/tcrain","followers_url":"https://api.github.com/users/tcrain/followers","following_url":"https://api.github.com/users/tcrain/following{/other_user}","gists_url":"https://api.github.com/users/tcrain/gists{/gist_id}","starred_url":"https://api.github.com/users/tcrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcrain/subscriptions","organizations_url":"https://api.github.com/users/tcrain/orgs","repos_url":"https://api.github.com/users/tcrain/repos","events_url":"https://api.github.com/users/tcrain/events{/privacy}","received_events_url":"https://api.github.com/users/tcrain/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"cmeiklejohn","id":44939,"avatar_url":"https://avatars.githubusercontent.com/u/44939?v=3","gravatar_id":"","url":"https://api.github.com/users/cmeiklejohn","html_url":"https://github.com/cmeiklejohn","followers_url":"https://api.github.com/users/cmeiklejohn/followers","following_url":"https://api.github.com/users/cmeiklejohn/following{/other_user}","gists_url":"https://api.github.com/users/cmeiklejohn/gists{/gist_id}","starred_url":"https://api.github.com/users/cmeiklejohn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmeiklejohn/subscriptions","organizations_url":"https://api.github.com/users/cmeiklejohn/orgs","repos_url":"https://api.github.com/users/cmeiklejohn/repos","events_url":"https://api.github.com/users/cmeiklejohn/events{/privacy}","received_events_url":"https://api.github.com/users/cmeiklejohn/received_events","type":"User","site_admin":false},"milestone":null,"comments":18,"created_at":"2015-06-26T16:27:01Z","updated_at":"2015-09-03T15:34:49Z","closed_at":"2015-09-01T15:24:02Z","pull_request":{"url":"https://api.github.com/repos/SyncFree/antidote/pulls/139","html_url":"https://github.com/SyncFree/antidote/pull/139","diff_url":"https://github.com/SyncFree/antidote/pull/139.diff","patch_url":"https://github.com/SyncFree/antidote/pull/139.patch"},"body":"Sorry for the size of this pull request, it is a result of merging with another largely refactored branch, future ones will be smaller.  If there are certain changes that you feel are bad or buggy or not necessary, let me know and I will remove them.  These changes are broken down into three main components, described in the following:\r\n\r\n1: Materialiser fixes/optimisations: This update fixes an issue where in certain cases garbage collection of old versions would cause updates to be lost, as well as issues where snapshots could be generated incorrectly when multiple DCs were performing updates.  Unit tests are added to check this.  Optimisations were done to allow for different snapshots to be materialised concurrently as well as storing the snapshots and operations ordered so that only enough of them are processed to be safe when generating new snapshots.  An new dictionary data structure is created for this that (partially) orders elements by vector clocks.  New snapshots are only stored if they are more recent than what is previously available, the reasoning behind this is that new transactions are started with the most recent available snapshot, so old snapshots probably won't be accessed often, though of course this depends on the workload.\r\n\r\n2: Read/write path optimisations: The goal of these changes is to avoid unnecessary serialisation points and bottlenecks.  For example currently a read goes through the transaction coordinator, and then is serialised at the clocksi vnode where a read fsm is created, and then is serialised again at the materialiser vnode where a snapshot is generated.  This branch changes that so the transaction coordinator sends a request to a read server running at a vnode which directly processes the updates from the ets tables.  Each vnode has a configurable number of concurrent read servers.  This can be especially helpful for keys that have really busy read workloads.  Of course this also means we have to be careful with updates being prepared concurrently, requiring the prepared table to be written to twice during commit to ensure transaction atomicity.  Single read operations send their request directly to the read server instead of first starting up a transaction coordinator. Because of these changes, before performing reads the system needs to check that the vnodes have been initialised, the check_ready functions provide this functionality. \r\n\r\nConcerning writes, they are now cached in the transaction coordinator then sent to clocksi vnode during prepare, removing another point of serialisation.  Logging the updates happens during the prepare phase, though I might suggest moving the logging back to when the updated is executed in the transaction since it can be done asynchronously.  There are still many more optimisations to be done such as blind writes and caching reads, but these can be done in future pull requests.\r\n\r\n3: Merging with evaluation branch:  This branch is merged into the evaluation branch, which included many changes from other people.  I think the main changes in this concern improvements to the transaction coordinator.   For example, the coordinator now chooses what method a transaction will use to commit based on the number of sites it updates.  For example if the transaction only updates a single site the prepare and commit can happen in a single round.  To keep with the current interface, the client still has to first call prepare then commit.  This branch also changes the logging vnode so that is uses erlang disk log instead of dets.\r\n\r\nThis branch still uses the old validation check for ensuring per dc snapshot isolation. Li has created a new version of this in a separate branch which will be a later pull request.\r\n\r\nPS I cannot get the pb_client_test to run, due to some dependency error, maybe someone else has the same problem?"},"comment":{"url":"https://api.github.com/repos/SyncFree/antidote/issues/comments/137487372","html_url":"https://github.com/SyncFree/antidote/pull/139#issuecomment-137487372","issue_url":"https://api.github.com/repos/SyncFree/antidote/issues/139","id":137487372,"user":{"login":"tcrain","id":5465025,"avatar_url":"https://avatars.githubusercontent.com/u/5465025?v=3","gravatar_id":"","url":"https://api.github.com/users/tcrain","html_url":"https://github.com/tcrain","followers_url":"https://api.github.com/users/tcrain/followers","following_url":"https://api.github.com/users/tcrain/following{/other_user}","gists_url":"https://api.github.com/users/tcrain/gists{/gist_id}","starred_url":"https://api.github.com/users/tcrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcrain/subscriptions","organizations_url":"https://api.github.com/users/tcrain/orgs","repos_url":"https://api.github.com/users/tcrain/repos","events_url":"https://api.github.com/users/tcrain/events{/privacy}","received_events_url":"https://api.github.com/users/tcrain/received_events","type":"User","site_admin":false},"created_at":"2015-09-03T15:34:49Z","updated_at":"2015-09-03T15:34:49Z","body":"Approx 8x on the left.\r\n\r\n2 is read latencies, 3 write latencies "}}