{"action":"closed","issue":{"url":"https://api.github.com/repos/IQSS/dataverse/issues/803","labels_url":"https://api.github.com/repos/IQSS/dataverse/issues/803/labels{/name}","comments_url":"https://api.github.com/repos/IQSS/dataverse/issues/803/comments","events_url":"https://api.github.com/repos/IQSS/dataverse/issues/803/events","html_url":"https://github.com/IQSS/dataverse/issues/803","id":39457275,"number":803,"title":"Batch Import: in v3.6 utility, seems not thread safe; weird errors can happen with large imports. ","user":{"login":"kcondon","id":4983816,"avatar_url":"https://avatars.githubusercontent.com/u/4983816?v=3","gravatar_id":"","url":"https://api.github.com/users/kcondon","html_url":"https://github.com/kcondon","followers_url":"https://api.github.com/users/kcondon/followers","following_url":"https://api.github.com/users/kcondon/following{/other_user}","gists_url":"https://api.github.com/users/kcondon/gists{/gist_id}","starred_url":"https://api.github.com/users/kcondon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kcondon/subscriptions","organizations_url":"https://api.github.com/users/kcondon/orgs","repos_url":"https://api.github.com/users/kcondon/repos","events_url":"https://api.github.com/users/kcondon/events{/privacy}","received_events_url":"https://api.github.com/users/kcondon/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/IQSS/dataverse/labels/Component%3A+Metadata","name":"Component: Metadata","color":"c7def8"},{"url":"https://api.github.com/repos/IQSS/dataverse/labels/Component%3A+UX+%26+Upgrade","name":"Component: UX & Upgrade","color":"c7def8"},{"url":"https://api.github.com/repos/IQSS/dataverse/labels/Status%3A+QA","name":"Status: QA","color":"336633"},{"url":"https://api.github.com/repos/IQSS/dataverse/labels/Type%3A+Bug","name":"Type: Bug","color":"fef2c0"}],"state":"closed","locked":false,"assignee":{"login":"kcondon","id":4983816,"avatar_url":"https://avatars.githubusercontent.com/u/4983816?v=3","gravatar_id":"","url":"https://api.github.com/users/kcondon","html_url":"https://github.com/kcondon","followers_url":"https://api.github.com/users/kcondon/followers","following_url":"https://api.github.com/users/kcondon/following{/other_user}","gists_url":"https://api.github.com/users/kcondon/gists{/gist_id}","starred_url":"https://api.github.com/users/kcondon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kcondon/subscriptions","organizations_url":"https://api.github.com/users/kcondon/orgs","repos_url":"https://api.github.com/users/kcondon/repos","events_url":"https://api.github.com/users/kcondon/events{/privacy}","received_events_url":"https://api.github.com/users/kcondon/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/IQSS/dataverse/milestones/22","labels_url":"https://api.github.com/repos/IQSS/dataverse/milestones/22/labels","id":891857,"number":22,"title":"Beta 11 - Dataverse 4.0","description":"-Bug fixing","creator":{"login":"eaquigley","id":5922904,"avatar_url":"https://avatars.githubusercontent.com/u/5922904?v=3","gravatar_id":"","url":"https://api.github.com/users/eaquigley","html_url":"https://github.com/eaquigley","followers_url":"https://api.github.com/users/eaquigley/followers","following_url":"https://api.github.com/users/eaquigley/following{/other_user}","gists_url":"https://api.github.com/users/eaquigley/gists{/gist_id}","starred_url":"https://api.github.com/users/eaquigley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eaquigley/subscriptions","organizations_url":"https://api.github.com/users/eaquigley/orgs","repos_url":"https://api.github.com/users/eaquigley/repos","events_url":"https://api.github.com/users/eaquigley/events{/privacy}","received_events_url":"https://api.github.com/users/eaquigley/received_events","type":"User","site_admin":false},"open_issues":174,"closed_issues":24,"state":"open","created_at":"2014-12-04T18:42:45Z","updated_at":"2015-01-13T18:57:12Z","due_on":"2015-01-16T05:00:00Z","closed_at":null},"comments":2,"created_at":"2014-08-04T20:23:59Z","updated_at":"2015-01-13T18:57:12Z","closed_at":"2015-01-13T18:57:12Z","body":"Found during import of antislavery petitions.\r\nSeveral symptoms:  the set was imported twice for double the number of studies, or 90% were imported and the rest failed with the error: java.nio.channels.ClosedByInterruptException, or the studies were imported but were not fully indexed.\r\n\r\nImporting in batches of 200-300 works. \r\n\r\nRecording here to review for when we implement in v4.0.\r\n\r\nOK, some clarification on the doubling:\r\nIt seems browser-related. Small sets complete and update the UI with a success message. Sets of 200+ don't always update UI on complete and seems like the browser may resubmit request due to timeout or some such since if I close browser immediately after clicking batch import, this does not happen.\r\n\r\nAlso, index does not reliably happen at the end of large batch jobs and needs to be run manually or wait for nightly index.\r\n"}}