{"action":"closed","issue":{"url":"https://api.github.com/repos/blaze/castra/issues/44","labels_url":"https://api.github.com/repos/blaze/castra/issues/44/labels{/name}","comments_url":"https://api.github.com/repos/blaze/castra/issues/44/comments","events_url":"https://api.github.com/repos/blaze/castra/issues/44/events","html_url":"https://github.com/blaze/castra/issues/44","id":104119071,"number":44,"title":"__del__ method causes race condition between processes/threads","user":{"login":"jcrist","id":2783717,"avatar_url":"https://avatars.githubusercontent.com/u/2783717?v=3","gravatar_id":"","url":"https://api.github.com/users/jcrist","html_url":"https://github.com/jcrist","followers_url":"https://api.github.com/users/jcrist/followers","following_url":"https://api.github.com/users/jcrist/following{/other_user}","gists_url":"https://api.github.com/users/jcrist/gists{/gist_id}","starred_url":"https://api.github.com/users/jcrist/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcrist/subscriptions","organizations_url":"https://api.github.com/users/jcrist/orgs","repos_url":"https://api.github.com/users/jcrist/repos","events_url":"https://api.github.com/users/jcrist/events{/privacy}","received_events_url":"https://api.github.com/users/jcrist/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-31T19:32:15Z","updated_at":"2015-09-01T16:52:01Z","closed_at":"2015-09-01T16:52:01Z","body":"The `__del__` method causes a race condition between processes/threads reading from the same castra if it's initialized multiple times (different `Castra` objects, same filepath). Other workers can read from the file mid `flush`, which causes periodic failures (seen both locally, and on travis [here](https://travis-ci.org/blaze/dask/builds/78098031)).\r\n\r\nA few solutions:\r\n\r\n1. Add a `readonly` mode that doesn't do anything on `__del__`, and doesn't allow `extend`/`extend_sequence`. Could either be a `mode='r'` kwarg, or a `readonly=True` kwarg. Default would be writable, so no change in default behavior.\r\n\r\n2. Remove the automatic flush/delete behavior. I'm unsure if removing automatic `flush` as a default is worth it, but I'm definitely against automatic delete. IMO temporary files/directories should be managed externally to castra. This is a separate issue entirely though.\r\n\r\n3. File based locks to ensure only one writer to each file/no reading while writing to same file.\r\n\r\nMy preference is option 1."}}