{"action":"created","issue":{"url":"https://api.github.com/repos/boto/boto/issues/621","labels_url":"https://api.github.com/repos/boto/boto/issues/621/labels{/name}","comments_url":"https://api.github.com/repos/boto/boto/issues/621/comments","events_url":"https://api.github.com/repos/boto/boto/issues/621/events","html_url":"https://github.com/boto/boto/issues/621","id":3539239,"number":621,"title":"broken pipe error, non US-Standard region","user":{"login":"jkarneges","id":521953,"avatar_url":"https://avatars.githubusercontent.com/u/521953?v=3","gravatar_id":"","url":"https://api.github.com/users/jkarneges","html_url":"https://github.com/jkarneges","followers_url":"https://api.github.com/users/jkarneges/followers","following_url":"https://api.github.com/users/jkarneges/following{/other_user}","gists_url":"https://api.github.com/users/jkarneges/gists{/gist_id}","starred_url":"https://api.github.com/users/jkarneges/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jkarneges/subscriptions","organizations_url":"https://api.github.com/users/jkarneges/orgs","repos_url":"https://api.github.com/users/jkarneges/repos","events_url":"https://api.github.com/users/jkarneges/events{/privacy}","received_events_url":"https://api.github.com/users/jkarneges/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":36,"created_at":"2012-03-07T08:11:37Z","updated_at":"2015-04-27T12:43:25Z","closed_at":null,"body":"My S3 region is Northern California, aka us-west-1.  Uploads almost always fail after some timeout period, even if the files are \"small\" (<200K).  I'm using boto 2.2.2 and python 2.7.2.  The uploads are being made from an EC2 machine in the same region.\r\n\r\nHere's the exception I always get:\r\n\r\n```python\r\nTraceback (most recent call last):\r\n  File \"backup_to_s3.py\", line 25, in <module>\r\n    k.set_contents_from_filename(\"/tmp/%s\" % dumpfile)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 969, in set_contents_from_filename\r\n    encrypt_key=encrypt_key)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 902, in set_contents_from_file\r\n    size=size)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/key.py\", line 660, in send_file\r\n    query_args=query_args)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py\", line 449, in make_request\r\n    override_num_retries=override_num_retries)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/connection.py\", line 829, in make_request\r\n    return self._mexe(http_request, sender, override_num_retries)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/connection.py\", line 794, in _mexe\r\n    raise e\r\nsocket.error: [Errno 32] Broken pipe\r\n```\r\n\r\nI used is_secure=False and tcpdump to inspect what was happening.  It looks like by default requests from boto are sent to the primary S3 host, which then responds with redirects to the correct region using HTTP code 307.  While boto does honor the 307 for some exchanges, it seems not to for PUT, or at least not consistently, and so I think this is what is causing the failures.\r\n\r\nPassing host=\"s3-us-west-1.amazonaws.com\" to S3Connection's constructor solved the problem for me."},"comment":{"url":"https://api.github.com/repos/boto/boto/issues/comments/96636238","html_url":"https://github.com/boto/boto/issues/621#issuecomment-96636238","issue_url":"https://api.github.com/repos/boto/boto/issues/621","id":96636238,"user":{"login":"netinept","id":1034255,"avatar_url":"https://avatars.githubusercontent.com/u/1034255?v=3","gravatar_id":"","url":"https://api.github.com/users/netinept","html_url":"https://github.com/netinept","followers_url":"https://api.github.com/users/netinept/followers","following_url":"https://api.github.com/users/netinept/following{/other_user}","gists_url":"https://api.github.com/users/netinept/gists{/gist_id}","starred_url":"https://api.github.com/users/netinept/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/netinept/subscriptions","organizations_url":"https://api.github.com/users/netinept/orgs","repos_url":"https://api.github.com/users/netinept/repos","events_url":"https://api.github.com/users/netinept/events{/privacy}","received_events_url":"https://api.github.com/users/netinept/received_events","type":"User","site_admin":false},"created_at":"2015-04-27T12:43:25Z","updated_at":"2015-04-27T12:43:25Z","body":"+1 This issue is still ongoing with python 2.7.6 and boto 2.38.0 on OS 10.10.3 and Heroku.\r\n\r\nFor me this issue always happens with jquery.js from `[project name]/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/jquery.js'`\r\n\r\nI've also found a recent [StackOverflow question](http://stackoverflow.com/questions/29552242/using-djangos-collectstatic-with-boto-s3-throws-error-32-broken-pipe-after-a) with the same issue.\r\n\r\nI have tried setting the region and even creating a new bucket on the default region, but that still fails.\r\n\r\n```\r\n(venv)kylefalconer@Kyles-MacBook-Air ~/g/plog> python manage.py collectstatic\r\n\r\nYou have requested to collect static files at the destination\r\nlocation as specified in your settings.\r\n\r\nThis will overwrite existing files!\r\nAre you sure you want to do this?\r\n\r\nType 'yes' to continue, or 'no' to cancel: yes\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/base.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/changelists.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/dashboard.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/forms.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/ie.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/login.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/rtl.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/widgets.css'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/changelist-bg.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/changelist-bg_rtl.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/default-bg-reverse.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/default-bg.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/deleted-overlay.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon-no.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon-unknown.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon-yes.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_addlink.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_alert.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_calendar.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_changelink.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_clock.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_deletelink.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_error.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_searchbox.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/icon_success.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/inline-delete-8bit.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/inline-delete.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/inline-restore-8bit.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/inline-restore.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/inline-splitter-bg.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/nav-bg-grabber.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/nav-bg-reverse.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/nav-bg-selected.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/nav-bg.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/selector-icons.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/selector-search.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/sorting-icons.gif'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/tooltag-add.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/tooltag-arrowright.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/gis/move_vertex_off.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/img/gis/move_vertex_on.png'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/actions.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/actions.min.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/calendar.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/collapse.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/collapse.min.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/core.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/inlines.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/inlines.min.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/jquery.init.js'\r\nCopying '/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/jquery.js'\r\nTraceback (most recent call last):\r\n  File \"manage.py\", line 10, in <module>\r\n    execute_from_command_line(sys.argv)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/core/management/__init__.py\", line 338, in execute_from_command_line\r\n    utility.execute()\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/core/management/__init__.py\", line 330, in execute\r\n    self.fetch_command(subcommand).run_from_argv(self.argv)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/core/management/base.py\", line 390, in run_from_argv\r\n    self.execute(*args, **cmd_options)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/core/management/base.py\", line 441, in execute\r\n    output = self.handle(*args, **options)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 168, in handle\r\n    collected = self.collect()\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 107, in collect\r\n    handler(path, prefixed_path, storage)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py\", line 315, in copy_file\r\n    self.storage.save(prefixed_path, source_file)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/django/core/files/storage.py\", line 64, in save\r\n    name = self._save(name, content)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/storages/backends/s3boto.py\", line 392, in _save\r\n    self._save_content(key, content, headers=headers)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/storages/backends/s3boto.py\", line 403, in _save_content\r\n    rewind=True, **kwargs)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/s3/key.py\", line 1293, in set_contents_from_file\r\n    chunked_transfer=chunked_transfer, size=size)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/s3/key.py\", line 750, in send_file\r\n    chunked_transfer=chunked_transfer, size=size)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/s3/key.py\", line 951, in _send_file_internal\r\n    query_args=query_args\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/s3/connection.py\", line 664, in make_request\r\n    retry_handler=retry_handler\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/connection.py\", line 1071, in make_request\r\n    retry_handler=retry_handler)\r\n  File \"/Users/kylefalconer/git/plog/venv/lib/python2.7/site-packages/boto/connection.py\", line 1030, in _mexe\r\n    raise ex\r\nsocket.error: [Errno 32] Broken pipe\r\n(venv)kylefalconer@Kyles-MacBook-Air ~/g/plog> touch .boto\r\n```"}}