{"action":"created","issue":{"url":"https://api.github.com/repos/snowplow/snowplow/issues/1647","labels_url":"https://api.github.com/repos/snowplow/snowplow/issues/1647/labels{/name}","comments_url":"https://api.github.com/repos/snowplow/snowplow/issues/1647/comments","events_url":"https://api.github.com/repos/snowplow/snowplow/issues/1647/events","html_url":"https://github.com/snowplow/snowplow/issues/1647","id":72226675,"number":1647,"title":"Scala S3 Sink?: S3 permissions problem when enriching in EMR","user":{"login":"alexanderdean","id":200511,"avatar_url":"https://avatars.githubusercontent.com/u/200511?v=3","gravatar_id":"","url":"https://api.github.com/users/alexanderdean","html_url":"https://github.com/alexanderdean","followers_url":"https://api.github.com/users/alexanderdean/followers","following_url":"https://api.github.com/users/alexanderdean/following{/other_user}","gists_url":"https://api.github.com/users/alexanderdean/gists{/gist_id}","starred_url":"https://api.github.com/users/alexanderdean/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexanderdean/subscriptions","organizations_url":"https://api.github.com/users/alexanderdean/orgs","repos_url":"https://api.github.com/users/alexanderdean/repos","events_url":"https://api.github.com/users/alexanderdean/events{/privacy}","received_events_url":"https://api.github.com/users/alexanderdean/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/snowplow/snowplow/labels/3.%20ETL","name":"3. ETL","color":"c3d69b"},{"url":"https://api.github.com/repos/snowplow/snowplow/labels/4.%20Storage","name":"4. Storage","color":"fcd37b"},{"url":"https://api.github.com/repos/snowplow/snowplow/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"alexanderdean","id":200511,"avatar_url":"https://avatars.githubusercontent.com/u/200511?v=3","gravatar_id":"","url":"https://api.github.com/users/alexanderdean","html_url":"https://github.com/alexanderdean","followers_url":"https://api.github.com/users/alexanderdean/followers","following_url":"https://api.github.com/users/alexanderdean/following{/other_user}","gists_url":"https://api.github.com/users/alexanderdean/gists{/gist_id}","starred_url":"https://api.github.com/users/alexanderdean/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexanderdean/subscriptions","organizations_url":"https://api.github.com/users/alexanderdean/orgs","repos_url":"https://api.github.com/users/alexanderdean/repos","events_url":"https://api.github.com/users/alexanderdean/events{/privacy}","received_events_url":"https://api.github.com/users/alexanderdean/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/snowplow/snowplow/milestones/76","html_url":"https://github.com/snowplow/snowplow/milestones/Release%2065%20Scarlet%20Rosefinch","labels_url":"https://api.github.com/repos/snowplow/snowplow/milestones/76/labels","id":948749,"number":76,"title":"Release 65 Scarlet Rosefinch","description":"","creator":{"login":"alexanderdean","id":200511,"avatar_url":"https://avatars.githubusercontent.com/u/200511?v=3","gravatar_id":"","url":"https://api.github.com/users/alexanderdean","html_url":"https://github.com/alexanderdean","followers_url":"https://api.github.com/users/alexanderdean/followers","following_url":"https://api.github.com/users/alexanderdean/following{/other_user}","gists_url":"https://api.github.com/users/alexanderdean/gists{/gist_id}","starred_url":"https://api.github.com/users/alexanderdean/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexanderdean/subscriptions","organizations_url":"https://api.github.com/users/alexanderdean/orgs","repos_url":"https://api.github.com/users/alexanderdean/repos","events_url":"https://api.github.com/users/alexanderdean/events{/privacy}","received_events_url":"https://api.github.com/users/alexanderdean/received_events","type":"User","site_admin":false},"open_issues":44,"closed_issues":8,"state":"open","created_at":"2015-01-23T17:15:47Z","updated_at":"2015-04-30T17:25:48Z","due_on":null,"closed_at":null},"comments":7,"created_at":"2015-04-30T17:25:48Z","updated_at":"2015-05-01T12:30:56Z","closed_at":null,"body":"See this thread: https://groups.google.com/forum/#!topic/snowplow-user/nLjOuKwZoIQ\r\n\r\nWe have successfully reproduced this on snplow5 with lzo s3 sink and EMR enrichment. Notes:\r\n\r\n* We are running in a subnet, unlike Michael\r\n* Have tried deleting the .lzo.index files and running, and it still fails - suggests the problem isn't to do with reading the .lzo.index files\r\n\r\nNext test: run the snplow5 EMR process with some vanilla CloudFront Collector files to see if it's something magical about the .lzo files. "},"comment":{"url":"https://api.github.com/repos/snowplow/snowplow/issues/comments/98123664","html_url":"https://github.com/snowplow/snowplow/issues/1647#issuecomment-98123664","issue_url":"https://api.github.com/repos/snowplow/snowplow/issues/1647","id":98123664,"user":{"login":"alexanderdean","id":200511,"avatar_url":"https://avatars.githubusercontent.com/u/200511?v=3","gravatar_id":"","url":"https://api.github.com/users/alexanderdean","html_url":"https://github.com/alexanderdean","followers_url":"https://api.github.com/users/alexanderdean/followers","following_url":"https://api.github.com/users/alexanderdean/following{/other_user}","gists_url":"https://api.github.com/users/alexanderdean/gists{/gist_id}","starred_url":"https://api.github.com/users/alexanderdean/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexanderdean/subscriptions","organizations_url":"https://api.github.com/users/alexanderdean/orgs","repos_url":"https://api.github.com/users/alexanderdean/repos","events_url":"https://api.github.com/users/alexanderdean/events{/privacy}","received_events_url":"https://api.github.com/users/alexanderdean/received_events","type":"User","site_admin":false},"created_at":"2015-05-01T12:30:56Z","updated_at":"2015-05-01T12:30:56Z","body":"Here is more information on this: http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-iam-roles-calling.html\r\n\r\nWe can probably fix this by adding an S3DistCp step in front of the lzo files, so that hadoop-lzo doesn't have to attempt S3 access..."}}