{"action":"created","issue":{"url":"https://api.github.com/repos/boto/botocore/issues/577","labels_url":"https://api.github.com/repos/boto/botocore/issues/577/labels{/name}","comments_url":"https://api.github.com/repos/boto/botocore/issues/577/comments","events_url":"https://api.github.com/repos/boto/botocore/issues/577/events","html_url":"https://github.com/boto/botocore/issues/577","id":89298749,"number":577,"title":"ValueError: Unknown component: endpoint_resolver","user":{"login":"russellneufeld","id":475923,"avatar_url":"https://avatars.githubusercontent.com/u/475923?v=3","gravatar_id":"","url":"https://api.github.com/users/russellneufeld","html_url":"https://github.com/russellneufeld","followers_url":"https://api.github.com/users/russellneufeld/followers","following_url":"https://api.github.com/users/russellneufeld/following{/other_user}","gists_url":"https://api.github.com/users/russellneufeld/gists{/gist_id}","starred_url":"https://api.github.com/users/russellneufeld/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/russellneufeld/subscriptions","organizations_url":"https://api.github.com/users/russellneufeld/orgs","repos_url":"https://api.github.com/users/russellneufeld/repos","events_url":"https://api.github.com/users/russellneufeld/events{/privacy}","received_events_url":"https://api.github.com/users/russellneufeld/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-06-18T13:47:05Z","updated_at":"2015-08-31T18:41:08Z","closed_at":"2015-06-30T17:54:27Z","body":"I'm seeing a very sporadic error which I'm hoping someone here can explain.  See stack trace below.  This is happening less than 5 times a day on a site that's getting maybe a million requests a day, so it's pretty rare, but I'd like to understand what's going on and see if there's a way to fix it.  I should also mention that the EC2 instance this code runs on has an IAM role associated with it, and gets its creds automatically through that mechanism.\r\n\r\nAny idea what might be going wrong here?  Thanks.\r\n\r\n$ pip freeze | grep boto\r\nboto3==0.0.19\r\nbotocore==1.0.0b1\r\n\r\n  File \"/home/ubuntu/nasuni/nasuni/utils/aws/s3.py\", line 82, in list_bucket_contents\r\n    bucket = boto3.resource('s3', region_name=settings.AWS_REGION).Bucket(bucket)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto3/__init__.py\", line 87, in resource\r\n    return _get_default_session().resource(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto3/session.py\", line 270, in resource\r\n    aws_session_token=aws_session_token, config=config)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto3/session.py\", line 192, in client\r\n    aws_session_token=aws_session_token, config=config)\r\n  File \"/usr/local/lib/python2.7/dist-packages/botocore/session.py\", line 833, in create_client\r\n    endpoint_resolver = self.get_component('endpoint_resolver')\r\n  File \"/usr/local/lib/python2.7/dist-packages/botocore/session.py\", line 741, in get_component\r\n    return self._components.get_component(name)\r\n  File \"/usr/local/lib/python2.7/dist-packages/botocore/session.py\", line 857, in get_component\r\n    raise ValueError(\"Unknown component: %s\" % name)\r\nValueError: Unknown component: endpoint_resolver\r\n"},"comment":{"url":"https://api.github.com/repos/boto/botocore/issues/comments/136459707","html_url":"https://github.com/boto/botocore/issues/577#issuecomment-136459707","issue_url":"https://api.github.com/repos/boto/botocore/issues/577","id":136459707,"user":{"login":"russellneufeld","id":475923,"avatar_url":"https://avatars.githubusercontent.com/u/475923?v=3","gravatar_id":"","url":"https://api.github.com/users/russellneufeld","html_url":"https://github.com/russellneufeld","followers_url":"https://api.github.com/users/russellneufeld/followers","following_url":"https://api.github.com/users/russellneufeld/following{/other_user}","gists_url":"https://api.github.com/users/russellneufeld/gists{/gist_id}","starred_url":"https://api.github.com/users/russellneufeld/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/russellneufeld/subscriptions","organizations_url":"https://api.github.com/users/russellneufeld/orgs","repos_url":"https://api.github.com/users/russellneufeld/repos","events_url":"https://api.github.com/users/russellneufeld/events{/privacy}","received_events_url":"https://api.github.com/users/russellneufeld/received_events","type":"User","site_admin":false},"created_at":"2015-08-31T18:41:08Z","updated_at":"2015-08-31T18:41:08Z","body":"For what it's worth, I think this is the code with the thread safety issue (from botocore/session.py):\r\n\r\n```\r\n    def get_component(self, name):\r\n        if name in self._deferred:\r\n            factory = self._deferred.pop(name)\r\n            self._components[name] = factory()\r\n        try:\r\n            return self._components[name]\r\n        except KeyError:\r\n            raise ValueError(\"Unknown component: %s\" % name)\r\n```\r\n\r\nIf two threads evaluate the \"name in\" check to True, only one will successfully pop on the next line.  The other will fail.  Seems like putting a lock around this section would fix things.\r\n\r\nIn the meantime I'm putting this in my startup code under a lock:\r\n```\r\nboto3.setup_default_session()\r\nboto3.DEFAULT_SESSION._session.get_component('data_loader')\r\nboto3.DEFAULT_SESSION._session.get_component('event_emitter')\r\nboto3.DEFAULT_SESSION._session.get_component('endpoint_resolver')\r\nboto3.DEFAULT_SESSION._session.get_component('credential_provider')\r\n```"}}