{"action":"closed","issue":{"url":"https://api.github.com/repos/saltstack/salt/issues/24048","labels_url":"https://api.github.com/repos/saltstack/salt/issues/24048/labels{/name}","comments_url":"https://api.github.com/repos/saltstack/salt/issues/24048/comments","events_url":"https://api.github.com/repos/saltstack/salt/issues/24048/events","html_url":"https://github.com/saltstack/salt/issues/24048","id":79151345,"number":24048,"title":"HTTP 599 error on large file sources with new tornado default","user":{"login":"bradthurber","id":3045456,"avatar_url":"https://avatars.githubusercontent.com/u/3045456?v=3","gravatar_id":"","url":"https://api.github.com/users/bradthurber","html_url":"https://github.com/bradthurber","followers_url":"https://api.github.com/users/bradthurber/followers","following_url":"https://api.github.com/users/bradthurber/following{/other_user}","gists_url":"https://api.github.com/users/bradthurber/gists{/gist_id}","starred_url":"https://api.github.com/users/bradthurber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bradthurber/subscriptions","organizations_url":"https://api.github.com/users/bradthurber/orgs","repos_url":"https://api.github.com/users/bradthurber/repos","events_url":"https://api.github.com/users/bradthurber/events{/privacy}","received_events_url":"https://api.github.com/users/bradthurber/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/saltstack/salt/labels/Beryllium","name":"Beryllium","color":"fbca04"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Blocker","name":"Blocker","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Bug","name":"Bug","color":"e11d21"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Confirmed","name":"Confirmed","color":"c7def8"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Core","name":"Core","color":"fef2c0"},{"url":"https://api.github.com/repos/saltstack/salt/labels/Fixed%20Pending%20Verification","name":"Fixed Pending Verification","color":"d4c5f9"},{"url":"https://api.github.com/repos/saltstack/salt/labels/High%20Severity","name":"High Severity","color":"ff9143"},{"url":"https://api.github.com/repos/saltstack/salt/labels/P2","name":"P2","color":"155fb5"}],"state":"closed","locked":false,"assignee":{"login":"DmitryKuzmenko","id":4807387,"avatar_url":"https://avatars.githubusercontent.com/u/4807387?v=3","gravatar_id":"","url":"https://api.github.com/users/DmitryKuzmenko","html_url":"https://github.com/DmitryKuzmenko","followers_url":"https://api.github.com/users/DmitryKuzmenko/followers","following_url":"https://api.github.com/users/DmitryKuzmenko/following{/other_user}","gists_url":"https://api.github.com/users/DmitryKuzmenko/gists{/gist_id}","starred_url":"https://api.github.com/users/DmitryKuzmenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DmitryKuzmenko/subscriptions","organizations_url":"https://api.github.com/users/DmitryKuzmenko/orgs","repos_url":"https://api.github.com/users/DmitryKuzmenko/repos","events_url":"https://api.github.com/users/DmitryKuzmenko/events{/privacy}","received_events_url":"https://api.github.com/users/DmitryKuzmenko/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/saltstack/salt/milestones/62","html_url":"https://github.com/saltstack/salt/milestones/Be%200","labels_url":"https://api.github.com/repos/saltstack/salt/milestones/62/labels","id":1232054,"number":62,"title":"Be 0","description":"","creator":{"login":"ssgward","id":8439595,"avatar_url":"https://avatars.githubusercontent.com/u/8439595?v=3","gravatar_id":"","url":"https://api.github.com/users/ssgward","html_url":"https://github.com/ssgward","followers_url":"https://api.github.com/users/ssgward/followers","following_url":"https://api.github.com/users/ssgward/following{/other_user}","gists_url":"https://api.github.com/users/ssgward/gists{/gist_id}","starred_url":"https://api.github.com/users/ssgward/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ssgward/subscriptions","organizations_url":"https://api.github.com/users/ssgward/orgs","repos_url":"https://api.github.com/users/ssgward/repos","events_url":"https://api.github.com/users/ssgward/events{/privacy}","received_events_url":"https://api.github.com/users/ssgward/received_events","type":"User","site_admin":false},"open_issues":24,"closed_issues":42,"state":"open","created_at":"2015-07-29T17:40:37Z","updated_at":"2015-08-18T14:20:14Z","due_on":"2015-08-17T06:00:00Z","closed_at":null},"comments":10,"created_at":"2015-05-21T20:34:35Z","updated_at":"2015-08-18T14:20:14Z","closed_at":"2015-08-18T14:20:14Z","body":"Using latest ___develop___ branch (post 2015.5) will cause ```HTTP 599 ContentLength too long error``` with large files (>100MB?). For example - this command with a large file (160MB) will cause the error.:\r\n\r\nsalt '*develop*' cp.get_url https://s3.amazonaws.com/karpractice/jboss-eap-6.4.0.zip /tmp/blah\r\n\r\nTraced back to /salt/utils/http.py line \r\n```\r\n...\r\n382       result = HTTPClient().fetch(\r\n...\r\n```\r\n\r\nThis appears to be due to new use of tornado as default http client in http.query and the default  tornado limits for max_buffer_size (100MB?). As a workaround, adding the max_buffer_size argument will allow a larger file to download. An example that allows for 300MB:\r\n\r\n```\r\n...\r\n382       result = HTTPClient(max_buffer_size=1024*1024*300).fetch(\r\n...\r\n```\r\n\r\nIt should be noted that this not only impacts cp.get_url, but modules that use cp.get_url - such as archive.extracted\r\n\r\nPerhaps we need to be able to configure a download file size max now? I'm not sure how this impacts memory usage."}}