{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-cli/issues/976","labels_url":"https://api.github.com/repos/aws/aws-cli/issues/976/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-cli/issues/976/comments","events_url":"https://api.github.com/repos/aws/aws-cli/issues/976/events","html_url":"https://github.com/aws/aws-cli/issues/976","id":47090519,"number":976,"title":"\"cloudsearchdomain search\" generates invalid signature, 'upload-documents' works","user":{"login":"TiVoMaker","id":5116271,"avatar_url":"https://avatars.githubusercontent.com/u/5116271?v=3","gravatar_id":"","url":"https://api.github.com/users/TiVoMaker","html_url":"https://github.com/TiVoMaker","followers_url":"https://api.github.com/users/TiVoMaker/followers","following_url":"https://api.github.com/users/TiVoMaker/following{/other_user}","gists_url":"https://api.github.com/users/TiVoMaker/gists{/gist_id}","starred_url":"https://api.github.com/users/TiVoMaker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TiVoMaker/subscriptions","organizations_url":"https://api.github.com/users/TiVoMaker/orgs","repos_url":"https://api.github.com/users/TiVoMaker/repos","events_url":"https://api.github.com/users/TiVoMaker/events{/privacy}","received_events_url":"https://api.github.com/users/TiVoMaker/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2014-10-28T22:36:55Z","updated_at":"2015-01-12T23:41:02Z","closed_at":null,"body":"Latest awscli (1.5.4). I first use \"upload-documents\" to my search domain, and it works:\r\n---\r\n['aws', '--region', 'us-west-2', 'cloudsearchdomain', '--endpoint-url', u'http://doc-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com', 'upload-documents', '--content-type', 'application/json', '--documents', '/var/folders/r9/78dky0cj71v1dt8g71qft7wm0000gn/T/tmplT0ZDR']\r\n(endpoints obscured on purpose)\r\n---\r\nDebug output:\r\nlogtime=\"2014-10-28 14:31:16,302\" module=\"boto\" level=\"DEBUG\" Using access key found in environment variable.\r\nlogtime=\"2014-10-28 14:31:16,302\" module=\"boto\" level=\"DEBUG\" Using secret key found in environment variable.\r\nlogtime=\"2014-10-28 14:31:16,302\" module=\"boto\" level=\"DEBUG\" Method: POST\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Path: /\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Data: \r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Headers: {}\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Host: cloudsearch.us-west-2.amazonaws.com\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Port: 443\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Params: {'Action': 'DescribeDomains', 'Version': '2013-01-01', 'ContentType': 'JSON', 'DomainNames.member.1': 'context-labs-2'}\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" establishing HTTPS connection: host=cloudsearch.us-west-2.amazonaws.com, kwargs={'port': 443, 'timeout': 70}\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" Token: None\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" CanonicalRequest:\r\nPOST\r\n/\r\n\r\nhost:cloudsearch.us-west-2.amazonaws.com\r\nx-amz-date:20141028T213116Z\r\n\r\nhost;x-amz-date\r\nda0c116c28d4e60a54b9796c032e16262242fddf4f846719f84250bf7432222e\r\nlogtime=\"2014-10-28 14:31:16,303\" module=\"boto\" level=\"DEBUG\" StringToSign:\r\nAWS4-HMAC-SHA256\r\n20141028T213116Z\r\n20141028/us-west-2/cloudsearch/aws4_request\r\nfe958485e5203e203ab3447cbb47060d57156b4e593e3e0d3382c18698742670\r\nlogtime=\"2014-10-28 14:31:16,304\" module=\"boto\" level=\"DEBUG\" Signature:\r\n0993b20ab5e7bdb5f6f7ef36c58602f3daea0833f50f268a4c0a13de1d023045\r\nlogtime=\"2014-10-28 14:31:16,304\" module=\"boto\" level=\"DEBUG\" Final headers: {'Content-Length': '94', 'User-Agent': 'Boto/2.33.0 Python/2.7.6 Darwin/14.0.0', 'Host': u'cloudsearch.us-west-2.amazonaws.com', 'X-Amz-Date': '20141028T213116Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Authorization': u'AWS4-HMAC-SHA256 Credential=AKIAJXFDHKBFBTCGDBGA/20141028/us-west-2/cloudsearch/aws4_request,SignedHeaders=host;x-amz-date,Signature=0993b20ab5e7bdb5f6f7ef36c58602f3daea0833f50f268a4c0a13de1d023045'}\r\nlogtime=\"2014-10-28 14:31:16,447\" module=\"boto\" level=\"DEBUG\" wrapping ssl socket; CA certificate file=/Users/jmb/depots/stuff0/.venv/lib/python2.7/site-packages/boto/cacerts/cacerts.txt\r\nlogtime=\"2014-10-28 14:31:16,534\" module=\"boto\" level=\"DEBUG\" validating server certificate: hostname=cloudsearch.us-west-2.amazonaws.com, certificate hosts=['cloudsearch.us-west-2.amazonaws.com']\r\nlogtime=\"2014-10-28 14:31:16,712\" module=\"boto\" level=\"DEBUG\" Response headers: [('x-amzn-requestid', 'bfc46e82-5ee9-11e4-a8af-8f18f1271d33'), ('date', 'Tue, 28 Oct 2014 21:31:16 GMT'), ('content-length', '755'), ('content-type', 'application/json')]\r\nlogtime=\"2014-10-28 14:31:16,712\" module=\"boto\" level=\"DEBUG\" {\"DescribeDomainsResponse\":{\"DescribeDomainsResult\":{\"DomainStatusList\":[{\"ARN\":\"arn:aws:cloudsearch:us-west-2:351790146037:domain/context-labs-2\",\"Created\":true,\"Deleted\":false,\"DocService\":{\"Endpoint\":\"doc-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com\"},\"DomainId\":\"351790146037/context-labs-2\",\"DomainName\":\"context-labs-2\",\"Limits\":{\"MaximumPartitionCount\":10,\"MaximumReplicationCount\":5},\"Processing\":false,\"RequiresIndexDocuments\":false,\"SearchInstanceCount\":1,\"SearchInstanceType\":\"search.m1.small\",\"SearchPartitionCount\":1,\"SearchService\":{\"Endpoint\":\"search-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com\"}}]},\"ResponseMetadata\":{\"RequestId\":\"bfc46e82-5ee9-11e4-a8af-8f18f1271d33\"}}}\r\n\r\n========\r\nSame keys, same terminal window, 'search' fails:\r\n---\r\n['aws', '--region', 'us-west-2', 'cloudsearchdomain', '--endpoint-url', u'https://search-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com', '--output', 'json', 'search', '--query-parser', 'structured', '--filter-query', u\"(and user_id:'user_id:1')\", '--size', '20', '--return', '_all_fields,_score', '--sort', '_score desc', '--search-query', u\"(and 'john')\"]\r\n---\r\nDebug output:\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Using access key found in environment variable.\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Using secret key found in environment variable.\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Method: POST\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Path: /\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Data: \r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Headers: {}\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Host: cloudsearch.us-west-2.amazonaws.com\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Port: 443\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Params: {'Action': 'DescribeDomains', 'Version': '2013-01-01', 'ContentType': 'JSON', 'DomainNames.member.1': 'context-labs-2'}\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" establishing HTTPS connection: host=cloudsearch.us-west-2.amazonaws.com, kwargs={'port': 443, 'timeout': 70}\r\nlogtime=\"2014-10-28 15:17:04,417\" module=\"boto\" level=\"DEBUG\" Token: None\r\nlogtime=\"2014-10-28 15:17:04,418\" module=\"boto\" level=\"DEBUG\" CanonicalRequest:\r\nPOST\r\n/\r\n\r\nhost:cloudsearch.us-west-2.amazonaws.com\r\nx-amz-date:20141028T221704Z\r\n\r\nhost;x-amz-date\r\nda0c116c28d4e60a54b9796c032e16262242fddf4f846719f84250bf7432222e\r\nlogtime=\"2014-10-28 15:17:04,418\" module=\"boto\" level=\"DEBUG\" StringToSign:\r\nAWS4-HMAC-SHA256\r\n20141028T221704Z\r\n20141028/us-west-2/cloudsearch/aws4_request\r\n85beb198ba5836119cbfbb60e95ad440c343fd545e3bd565a5be896e293534e4\r\nlogtime=\"2014-10-28 15:17:04,418\" module=\"boto\" level=\"DEBUG\" Signature:\r\na891e0bfa5bb2e6f388e7b912a86b2e7a6edf22915aebd58799d964796348eee\r\nlogtime=\"2014-10-28 15:17:04,418\" module=\"boto\" level=\"DEBUG\" Final headers: {'Content-Length': '94', 'User-Agent': 'Boto/2.33.0 Python/2.7.6 Darwin/14.0.0', 'Host': u'cloudsearch.us-west-2.amazonaws.com', 'X-Amz-Date': '20141028T221704Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Authorization': u'AWS4-HMAC-SHA256 Credential=AKIAJXFDHKBFBTCGDBGA/20141028/us-west-2/cloudsearch/aws4_request,SignedHeaders=host;x-amz-date,Signature=a891e0bfa5bb2e6f388e7b912a86b2e7a6edf22915aebd58799d964796348eee'}\r\nlogtime=\"2014-10-28 15:17:04,504\" module=\"boto\" level=\"DEBUG\" wrapping ssl socket; CA certificate file=/Users/jmb/depots/stuff0/.venv/lib/python2.7/site-packages/boto/cacerts/cacerts.txt\r\nlogtime=\"2014-10-28 15:17:04,589\" module=\"boto\" level=\"DEBUG\" validating server certificate: hostname=cloudsearch.us-west-2.amazonaws.com, certificate hosts=['cloudsearch.us-west-2.amazonaws.com']\r\nlogtime=\"2014-10-28 15:17:04,740\" module=\"boto\" level=\"DEBUG\" Response headers: [('x-amzn-requestid', '25bc0bac-5ef0-11e4-8697-cb6a4f3fdb68'), ('date', 'Tue, 28 Oct 2014 22:17:04 GMT'), ('content-length', '754'), ('content-type', 'application/json')]\r\nlogtime=\"2014-10-28 15:17:04,740\" module=\"boto\" level=\"DEBUG\" {\"DescribeDomainsResponse\":{\"DescribeDomainsResult\":{\"DomainStatusList\":[{\"ARN\":\"arn:aws:cloudsearch:us-west-2:351790146037:domain/context-labs-2\",\"Created\":true,\"Deleted\":false,\"DocService\":{\"Endpoint\":\"doc-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com\"},\"DomainId\":\"351790146037/context-labs-2\",\"DomainName\":\"context-labs-2\",\"Limits\":{\"MaximumPartitionCount\":10,\"MaximumReplicationCount\":5},\"Processing\":true,\"RequiresIndexDocuments\":false,\"SearchInstanceCount\":1,\"SearchInstanceType\":\"search.m1.small\",\"SearchPartitionCount\":1,\"SearchService\":{\"Endpoint\":\"search-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com\"}}]},\"ResponseMetadata\":{\"RequestId\":\"25bc0bac-5ef0-11e4-8697-cb6a4f3fdb68\"}}}\r\nlogtime=\"2014-10-28 15:17:04,740\" module=\"search.cloudsearch_util\" level=\"INFO\" calling out to awscli: ['aws', '--region', 'us-west-2', 'cloudsearchdomain', '--endpoint-url', u'https://search-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com', '--output', 'json', 'search', '--query-parser', 'structured', '--filter-query', u\"(and user_id:'user_id:1')\", '--size', '20', '--return', '_all_fields,_score', '--sort', '_score desc', '--search-query', u\"(and 'john')\"]\r\n\r\nA client error (Unknown) occurred when calling the Search operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\r\n\r\nThe Canonical String for this request should have been\r\n'GET\r\n/2013-01-01/search\r\nformat=sdk&fq=%28and%20user_id%3A%27user_id%3A1%27%29&pretty=true&q=%28and%20%27john%27%29&q.parser=structured&return=_all_fields%2C_score&size=20&sort=_score%20desc\r\nhost:search-context-labs-2-2yzynxoc77jbuf4qrnwxbc5ssu.us-west-2.cloudsearch.amazonaws.com\r\nuser-agent:aws-cli/1.5.3 Python/2.7.6 Darwin/14.0.0\r\nx-amz-date:20141028T221705Z\r\n\r\nhost;user-agent;x-amz-date\r\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'\r\n\r\nThe String-to-Sign should have been\r\n'AWS4-HMAC-SHA256\r\n20141028T221705Z\r\n20141028/us-west-2/cloudsearch/aws4_request\r\n1f09ab269873c255c30f44f474c44c4397c6b75fcc802abfa8127763fbf30f99'\r\n==========\r\nIn case it helps any, I get the exact same failure mode when running on an EC2 instance with an instance profile set to allow full domain access. From the debug output, it finds the access token just fine, upload-documents works, search fails identically to above."},"comment":{"url":"https://api.github.com/repos/aws/aws-cli/issues/comments/69669399","html_url":"https://github.com/aws/aws-cli/issues/976#issuecomment-69669399","issue_url":"https://api.github.com/repos/aws/aws-cli/issues/976","id":69669399,"user":{"login":"catufunwa","id":302128,"avatar_url":"https://avatars.githubusercontent.com/u/302128?v=3","gravatar_id":"","url":"https://api.github.com/users/catufunwa","html_url":"https://github.com/catufunwa","followers_url":"https://api.github.com/users/catufunwa/followers","following_url":"https://api.github.com/users/catufunwa/following{/other_user}","gists_url":"https://api.github.com/users/catufunwa/gists{/gist_id}","starred_url":"https://api.github.com/users/catufunwa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/catufunwa/subscriptions","organizations_url":"https://api.github.com/users/catufunwa/orgs","repos_url":"https://api.github.com/users/catufunwa/repos","events_url":"https://api.github.com/users/catufunwa/events{/privacy}","received_events_url":"https://api.github.com/users/catufunwa/received_events","type":"User","site_admin":false},"created_at":"2015-01-12T23:41:02Z","updated_at":"2015-01-12T23:41:02Z","body":"Any update to this? I notice it happens if you use the --query-parser option it breaks the auth check.  "}}