{"action":"created","issue":{"url":"https://api.github.com/repos/arq5x/gemini/issues/406","labels_url":"https://api.github.com/repos/arq5x/gemini/issues/406/labels{/name}","comments_url":"https://api.github.com/repos/arq5x/gemini/issues/406/comments","events_url":"https://api.github.com/repos/arq5x/gemini/issues/406/events","html_url":"https://github.com/arq5x/gemini/issues/406","id":63771301,"number":406,"title":"Errors that cause load to fail with single core usage doesn't result in failure under multicore mode","user":{"login":"dgaston","id":1605762,"avatar_url":"https://avatars.githubusercontent.com/u/1605762?v=3","gravatar_id":"","url":"https://api.github.com/users/dgaston","html_url":"https://github.com/dgaston","followers_url":"https://api.github.com/users/dgaston/followers","following_url":"https://api.github.com/users/dgaston/following{/other_user}","gists_url":"https://api.github.com/users/dgaston/gists{/gist_id}","starred_url":"https://api.github.com/users/dgaston/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dgaston/subscriptions","organizations_url":"https://api.github.com/users/dgaston/orgs","repos_url":"https://api.github.com/users/dgaston/repos","events_url":"https://api.github.com/users/dgaston/events{/privacy}","received_events_url":"https://api.github.com/users/dgaston/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-23T16:44:57Z","updated_at":"2015-03-23T17:29:02Z","closed_at":null,"body":"I noticed after running into the ClinVar issue that @brentp posted about that when running gemini load with multiple cores while you still see the IndexErrors being thrown the load process may appear to complete. It is apparent it isn't completely successful as JSON images are left and not cleaned, and the database is too small, but for a user not paying close attention it may slip through the cracks. The main process doesn't appear to catch the errors and exit gracefully, and instead continues on to the merge step."},"comment":{"url":"https://api.github.com/repos/arq5x/gemini/issues/comments/85114105","html_url":"https://github.com/arq5x/gemini/issues/406#issuecomment-85114105","issue_url":"https://api.github.com/repos/arq5x/gemini/issues/406","id":85114105,"user":{"login":"brentp","id":1739,"avatar_url":"https://avatars.githubusercontent.com/u/1739?v=3","gravatar_id":"","url":"https://api.github.com/users/brentp","html_url":"https://github.com/brentp","followers_url":"https://api.github.com/users/brentp/followers","following_url":"https://api.github.com/users/brentp/following{/other_user}","gists_url":"https://api.github.com/users/brentp/gists{/gist_id}","starred_url":"https://api.github.com/users/brentp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brentp/subscriptions","organizations_url":"https://api.github.com/users/brentp/orgs","repos_url":"https://api.github.com/users/brentp/repos","events_url":"https://api.github.com/users/brentp/events{/privacy}","received_events_url":"https://api.github.com/users/brentp/received_events","type":"User","site_admin":false},"created_at":"2015-03-23T17:29:01Z","updated_at":"2015-03-23T17:29:01Z","body":"yes, I've seen this as well. If any of the cores have an error, we should fail sooner and remove the tmp files."}}