{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7121","labels_url":"https://api.github.com/repos/apache/spark/issues/7121/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7121/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7121/events","html_url":"https://github.com/apache/spark/pull/7121","id":92050576,"number":7121,"title":"[SPARK-8708][MLlib] Paritition ALS ratings based on both users and products","user":{"login":"viirya","id":68855,"avatar_url":"https://avatars.githubusercontent.com/u/68855?v=3","gravatar_id":"","url":"https://api.github.com/users/viirya","html_url":"https://github.com/viirya","followers_url":"https://api.github.com/users/viirya/followers","following_url":"https://api.github.com/users/viirya/following{/other_user}","gists_url":"https://api.github.com/users/viirya/gists{/gist_id}","starred_url":"https://api.github.com/users/viirya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/viirya/subscriptions","organizations_url":"https://api.github.com/users/viirya/orgs","repos_url":"https://api.github.com/users/viirya/repos","events_url":"https://api.github.com/users/viirya/events{/privacy}","received_events_url":"https://api.github.com/users/viirya/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-06-30T10:08:39Z","updated_at":"2015-06-30T15:03:44Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7121","html_url":"https://github.com/apache/spark/pull/7121","diff_url":"https://github.com/apache/spark/pull/7121.diff","patch_url":"https://github.com/apache/spark/pull/7121.patch"},"body":"JIRA: https://issues.apache.org/jira/browse/SPARK-8708\r\n\r\nPreviously the partitions of ratings are only based on the given products. So if the `usersProducts` given for prediction contains only few products or even one product, the generated ratings will be pushed into few or single partition and can't use high parallelism.\r\n\r\nThe following codes are the example reported in the JIRA. Because it asks the predictions for users on product 2. There is only one partition in the result.\r\n\r\n    >>> r1 = (1, 1, 1.0)\r\n    >>> r2 = (1, 2, 2.0)\r\n    >>> r3 = (2, 1, 2.0)\r\n    >>> r4 = (2, 2, 2.0)\r\n    >>> r5 = (3, 1, 1.0)\r\n    >>> ratings = sc.parallelize([r1, r2, r3, r4, r5], 5)\r\n    >>> users = ratings.map(itemgetter(0)).distinct()\r\n    >>> model = ALS.trainImplicit(ratings, 1, seed=10)\r\n    >>> predictions_for_2 = model.predictAll(users.map(lambda u: (u, 2)))\r\n    >>> predictions_for_2.glom().map(len).collect()\r\n    [0, 0, 3, 0, 0]\r\n\r\nThis PR uses user and product instead of only product to partition the ratings."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/117219689","html_url":"https://github.com/apache/spark/pull/7121#issuecomment-117219689","issue_url":"https://api.github.com/repos/apache/spark/issues/7121","id":117219689,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-06-30T15:03:44Z","updated_at":"2015-06-30T15:03:44Z","body":"  [Test build #36142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36142/consoleFull) for   PR 7121 at commit [`83e56c1`](https://github.com/apache/spark/commit/83e56c1627113e2493785cc9a429d3d7b97dc532)."}}