{"action":"created","issue":{"url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/3","labels_url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/3/comments","events_url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/3/events","html_url":"https://github.com/bozaro/git-lfs-migrate/issues/3","id":102466977,"number":3,"title":"Multi thread support causes problems with github lfs","user":{"login":"ManuelGysin","id":2873373,"avatar_url":"https://avatars.githubusercontent.com/u/2873373?v=3","gravatar_id":"","url":"https://api.github.com/users/ManuelGysin","html_url":"https://github.com/ManuelGysin","followers_url":"https://api.github.com/users/ManuelGysin/followers","following_url":"https://api.github.com/users/ManuelGysin/following{/other_user}","gists_url":"https://api.github.com/users/ManuelGysin/gists{/gist_id}","starred_url":"https://api.github.com/users/ManuelGysin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ManuelGysin/subscriptions","organizations_url":"https://api.github.com/users/ManuelGysin/orgs","repos_url":"https://api.github.com/users/ManuelGysin/repos","events_url":"https://api.github.com/users/ManuelGysin/events{/privacy}","received_events_url":"https://api.github.com/users/ManuelGysin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-08-21T21:39:45Z","updated_at":"2015-08-22T06:27:45Z","closed_at":null,"body":"Hello again! :)\r\n\r\nSince multi thread support is added I'm getting:\r\n\r\n```\r\nException in thread \"main\" java.io.IOException: Error writing request body to server\r\n\tat sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3478)\r\n\tat sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3461)\r\n\tat git.lfs.migrate.GitConverter.upload(GitConverter.java:320)\r\n\tat git.lfs.migrate.GitConverter.access$300(GitConverter.java:26)\r\n\tat git.lfs.migrate.GitConverter$5.convert(GitConverter.java:254)\r\n\tat git.lfs.migrate.Main.lambda$processMultipleThreads$0(Main.java:123)\r\n\tat git.lfs.migrate.Main$$Lambda$1/1059072014.run(Unknown Source)\r\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\r\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n\tSuppressed: java.io.IOException: insufficient data written\r\n\t\tat sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(HttpURLConnection.java:3500)\r\n\t\tat git.lfs.migrate.GitConverter.upload(GitConverter.java:322)\r\n\t\t... 9 more\r\n```\r\n\r\nand\r\n\r\n```\r\nException in thread \"main\" java.net.SocketException: Unexpected end of file from server\r\n\tat sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)\r\n\tat sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)\r\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)\r\n\tat git.lfs.migrate.GitConverter.upload(GitConverter.java:323)\r\n\tat git.lfs.migrate.GitConverter.access$300(GitConverter.java:26)\r\n\tat git.lfs.migrate.GitConverter$5.convert(GitConverter.java:254)\r\n\tat git.lfs.migrate.Main.lambda$processMultipleThreads$0(Main.java:123)\r\n\tat git.lfs.migrate.Main$$Lambda$1/1059072014.run(Unknown Source)\r\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\r\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n```\r\n\r\nI thinks this has something to do with API limitation on github or S3. The error occurs at random, sometimes 100 files can be uploaded, sometimes some thousands. \r\n\r\nI would suggest to reduce it to 4 threads as default, seems to run stable here. (currently testing with a 250Mbit/s upload connection)\r\n\r\nGreets\r\nManuel"},"comment":{"url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/comments/133644233","html_url":"https://github.com/bozaro/git-lfs-migrate/issues/3#issuecomment-133644233","issue_url":"https://api.github.com/repos/bozaro/git-lfs-migrate/issues/3","id":133644233,"user":{"login":"bozaro","id":2458138,"avatar_url":"https://avatars.githubusercontent.com/u/2458138?v=3","gravatar_id":"","url":"https://api.github.com/users/bozaro","html_url":"https://github.com/bozaro","followers_url":"https://api.github.com/users/bozaro/followers","following_url":"https://api.github.com/users/bozaro/following{/other_user}","gists_url":"https://api.github.com/users/bozaro/gists{/gist_id}","starred_url":"https://api.github.com/users/bozaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bozaro/subscriptions","organizations_url":"https://api.github.com/users/bozaro/orgs","repos_url":"https://api.github.com/users/bozaro/repos","events_url":"https://api.github.com/users/bozaro/events{/privacy}","received_events_url":"https://api.github.com/users/bozaro/received_events","type":"User","site_admin":false},"created_at":"2015-08-22T06:27:45Z","updated_at":"2015-08-22T06:27:45Z","body":"As workaround you also can limit thread count by ```--threads 1``` (default value: current CPU cores)."}}