{"action":"closed","issue":{"url":"https://api.github.com/repos/kennethreitz/requests/issues/2745","labels_url":"https://api.github.com/repos/kennethreitz/requests/issues/2745/labels{/name}","comments_url":"https://api.github.com/repos/kennethreitz/requests/issues/2745/comments","events_url":"https://api.github.com/repos/kennethreitz/requests/issues/2745/events","html_url":"https://github.com/kennethreitz/requests/issues/2745","id":102989781,"number":2745,"title":"Very slow read of large files compared to many other modules/binaries (e.g. urllib/curl/wget)","user":{"login":"davechallis","id":208316,"avatar_url":"https://avatars.githubusercontent.com/u/208316?v=3","gravatar_id":"","url":"https://api.github.com/users/davechallis","html_url":"https://github.com/davechallis","followers_url":"https://api.github.com/users/davechallis/followers","following_url":"https://api.github.com/users/davechallis/following{/other_user}","gists_url":"https://api.github.com/users/davechallis/gists{/gist_id}","starred_url":"https://api.github.com/users/davechallis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davechallis/subscriptions","organizations_url":"https://api.github.com/users/davechallis/orgs","repos_url":"https://api.github.com/users/davechallis/repos","events_url":"https://api.github.com/users/davechallis/events{/privacy}","received_events_url":"https://api.github.com/users/davechallis/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-08-25T09:50:34Z","updated_at":"2015-08-25T10:01:07Z","closed_at":"2015-08-25T10:01:07Z","body":"Using python 3.4 and requests 2.7.0 (on both Ubuntu 14.04 and Mac OS X), I've noticed that fetching raw content of large files is significantly slower (~5x) with requests than many other applications and python modules, performance as below is typical:\r\n\r\n    import requests                                                             \r\n    import urllib.request                                                       \r\n    import time                                                                 \r\n           \r\n    # returns a 100Mb JSON file                                                                     \r\n    url = 'http://some.host.on.local.network/'\r\n                                                                                \r\n    def with_urllib():                                                          \r\n        with urllib.request.urlopen(url) as f:                                  \r\n            return f.read()                                                     \r\n                                                                                \r\n    def with_requests():                                                        \r\n        return requests.get(url).content                                        \r\n                                                                                \r\n    s = time.time()                                                             \r\n    a = with_urllib()                                                           \r\n    print('urllib:', time.time() - s, 'seconds')                                \r\n                                                                                \r\n    s = time.time()                                                             \r\n    b = with_requests()                                                         \r\n    print('requests:', time.time() - s, 'seconds')                              \r\n                                                                                \r\n    assert a == b\r\n\r\nThis outputs:\r\n\r\n    urllib: 1.113746166229248 seconds\r\n    requests: 5.458410978317261 seconds\r\n\r\ncurl, wget, pycurl and urllib all complete in similar times (~1.1s on average), requests consistently takes ~5.4s on average, even with various combinations of parameters I've tried (e.g. with streaming, with different streaming chunksizes, etc.)."}}