{"action":"created","issue":{"url":"https://api.github.com/repos/basho/machi/issues/36","labels_url":"https://api.github.com/repos/basho/machi/issues/36/labels{/name}","comments_url":"https://api.github.com/repos/basho/machi/issues/36/comments","events_url":"https://api.github.com/repos/basho/machi/issues/36/events","html_url":"https://github.com/basho/machi/issues/36","id":114244162,"number":36,"title":"filename manager is reusing file names after an epoch change","user":{"login":"slfritchie","id":75320,"avatar_url":"https://avatars.githubusercontent.com/u/75320?v=3","gravatar_id":"","url":"https://api.github.com/users/slfritchie","html_url":"https://github.com/slfritchie","followers_url":"https://api.github.com/users/slfritchie/followers","following_url":"https://api.github.com/users/slfritchie/following{/other_user}","gists_url":"https://api.github.com/users/slfritchie/gists{/gist_id}","starred_url":"https://api.github.com/users/slfritchie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/slfritchie/subscriptions","organizations_url":"https://api.github.com/users/slfritchie/orgs","repos_url":"https://api.github.com/users/slfritchie/repos","events_url":"https://api.github.com/users/slfritchie/events{/privacy}","received_events_url":"https://api.github.com/users/slfritchie/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-10-30T10:38:19Z","updated_at":"2015-11-04T21:49:32Z","closed_at":null,"body":"This bug was originally found by @shino's work on #33 ... that QuickCheck model is working!  Shino's QuickCheck model almost always finds this error within a couple of minutes.\r\n\r\nDescription of an actual counter-example:\r\n\r\n* During epoch 1113, UPI=[a,b,c] ... and there is some data written to this file, and it is written successfully to all a,b,c.\r\n* Then epoch changes to 1115.  B believes that UPI=[b].\r\n* The first append to b, the epoch is different, so the filename_mgr uses the line 155 case, creates a new name (correctly, yay!).\r\n* The *second append* to b in epoch 1115, the handle_find_file() is ... looking on disk to find a file to append to, finds the the file written during epoch 1113, and uses it for this second append.  This is when Machi's eventual consistency \"files are always mergeable\" property fails.\r\n\r\nIn eventual consistency mode, the filename manager must always pick a new & unique file name after an epoch change.  In strong consistency mode, reuse after an epoch change is OK because we don't have the same merging requirement.\r\n\r\nHowever, we'd still like to preserve the ability to append to a file after a period of inactivity.  The current implementation (which uses `filelib:wildcard()` and so needs a refactoring sometime for better efficiency) appears to do that AFAICT.  So a modification that can do the right thing with eventual mode + epoch change seems best?"},"comment":{"url":"https://api.github.com/repos/basho/machi/issues/comments/153877158","html_url":"https://github.com/basho/machi/issues/36#issuecomment-153877158","issue_url":"https://api.github.com/repos/basho/machi/issues/36","id":153877158,"user":{"login":"mrallen1","id":204728,"avatar_url":"https://avatars.githubusercontent.com/u/204728?v=3","gravatar_id":"","url":"https://api.github.com/users/mrallen1","html_url":"https://github.com/mrallen1","followers_url":"https://api.github.com/users/mrallen1/followers","following_url":"https://api.github.com/users/mrallen1/following{/other_user}","gists_url":"https://api.github.com/users/mrallen1/gists{/gist_id}","starred_url":"https://api.github.com/users/mrallen1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mrallen1/subscriptions","organizations_url":"https://api.github.com/users/mrallen1/orgs","repos_url":"https://api.github.com/users/mrallen1/repos","events_url":"https://api.github.com/users/mrallen1/events{/privacy}","received_events_url":"https://api.github.com/users/mrallen1/received_events","type":"User","site_admin":false},"created_at":"2015-11-04T21:49:32Z","updated_at":"2015-11-04T21:49:32Z","body":"The current file sequence number is stored on disk, so it should be safe to remove files that have been completely trimmed. I think ultimately we need to **not** use the wildcard function (I copied that bit of code from the FLU itself) but as for what we should replace that with I'm not certain - there are a lot of options that might work ok."}}