{"action":"created","issue":{"url":"https://api.github.com/repos/Yelp/mrjob/issues/1066","labels_url":"https://api.github.com/repos/Yelp/mrjob/issues/1066/labels{/name}","comments_url":"https://api.github.com/repos/Yelp/mrjob/issues/1066/comments","events_url":"https://api.github.com/repos/Yelp/mrjob/issues/1066/events","html_url":"https://github.com/Yelp/mrjob/issues/1066","id":92130906,"number":1066,"title":"Support distributed cache","user":{"login":"tianhuil","id":1906321,"avatar_url":"https://avatars.githubusercontent.com/u/1906321?v=3","gravatar_id":"","url":"https://api.github.com/users/tianhuil","html_url":"https://github.com/tianhuil","followers_url":"https://api.github.com/users/tianhuil/followers","following_url":"https://api.github.com/users/tianhuil/following{/other_user}","gists_url":"https://api.github.com/users/tianhuil/gists{/gist_id}","starred_url":"https://api.github.com/users/tianhuil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tianhuil/subscriptions","organizations_url":"https://api.github.com/users/tianhuil/orgs","repos_url":"https://api.github.com/users/tianhuil/repos","events_url":"https://api.github.com/users/tianhuil/events{/privacy}","received_events_url":"https://api.github.com/users/tianhuil/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-30T15:55:56Z","updated_at":"2015-06-30T20:31:39Z","closed_at":"2015-06-30T19:31:36Z","body":"Currently, I believe there is no (easy) way to specify a job that normalizes feature values (e.g. for machine learning).  For example, if we have a file where each row is a value $X_k$ and we want\r\n$\\frac{X_k}{ \\sum_k X_k}$,\r\nthe only solution is to output the sum in a mapreduce and then feed this sum file and the original file into a second mapreduce (via a distributed cache mechanism).  This is also commonly used for map-side join.s\r\n\r\nSome simple support for reading and writing from the cache would be great!"},"comment":{"url":"https://api.github.com/repos/Yelp/mrjob/issues/comments/117332876","html_url":"https://github.com/Yelp/mrjob/issues/1066#issuecomment-117332876","issue_url":"https://api.github.com/repos/Yelp/mrjob/issues/1066","id":117332876,"user":{"login":"tianhuil","id":1906321,"avatar_url":"https://avatars.githubusercontent.com/u/1906321?v=3","gravatar_id":"","url":"https://api.github.com/users/tianhuil","html_url":"https://github.com/tianhuil","followers_url":"https://api.github.com/users/tianhuil/followers","following_url":"https://api.github.com/users/tianhuil/following{/other_user}","gists_url":"https://api.github.com/users/tianhuil/gists{/gist_id}","starred_url":"https://api.github.com/users/tianhuil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tianhuil/subscriptions","organizations_url":"https://api.github.com/users/tianhuil/orgs","repos_url":"https://api.github.com/users/tianhuil/repos","events_url":"https://api.github.com/users/tianhuil/events{/privacy}","received_events_url":"https://api.github.com/users/tianhuil/received_events","type":"User","site_admin":false},"created_at":"2015-06-30T20:31:39Z","updated_at":"2015-06-30T20:31:39Z","body":"Hi,\r\n\r\nI know this is certainly possible in Hadoop.  Is there a recommended way of mapping $X_k$ to\r\n$\\frac{X_k}{ \\sum_k X_k}$?\r\n\r\nMichael"}}