{"action":"created","issue":{"url":"https://api.github.com/repos/databricks/spark-perf/issues/46","labels_url":"https://api.github.com/repos/databricks/spark-perf/issues/46/labels{/name}","comments_url":"https://api.github.com/repos/databricks/spark-perf/issues/46/comments","events_url":"https://api.github.com/repos/databricks/spark-perf/issues/46/events","html_url":"https://github.com/databricks/spark-perf/issues/46","id":55297075,"number":46,"title":"Python mllib tests failing","user":{"login":"tsailiming","id":1571401,"avatar_url":"https://avatars.githubusercontent.com/u/1571401?v=3","gravatar_id":"","url":"https://api.github.com/users/tsailiming","html_url":"https://github.com/tsailiming","followers_url":"https://api.github.com/users/tsailiming/followers","following_url":"https://api.github.com/users/tsailiming/following{/other_user}","gists_url":"https://api.github.com/users/tsailiming/gists{/gist_id}","starred_url":"https://api.github.com/users/tsailiming/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tsailiming/subscriptions","organizations_url":"https://api.github.com/users/tsailiming/orgs","repos_url":"https://api.github.com/users/tsailiming/repos","events_url":"https://api.github.com/users/tsailiming/events{/privacy}","received_events_url":"https://api.github.com/users/tsailiming/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-01-23T15:51:47Z","updated_at":"2015-02-17T18:32:35Z","closed_at":"2015-02-17T18:32:35Z","body":"All tests are failing because of the random seed. \r\n\r\nNumber of failed tests: 8, failed tests: python-glm-classification,python-glm-classification,python-glm-regression,python-naive-bayes,python-als,python-kmeans,python-pearson,python-spearman\r\n\r\n```\r\n15/01/23 23:17:32 WARN scheduler.TaskSetManager: Lost task 11.0 in stage 0.0 (TID 11, numaq1-1): org.apache.spark.api.python.PythonException: Traceback (most recent call last):\r\n  File \"/opt/spark-1.2.0-bin-hadoop2.4/python/pyspark/worker.py\", line 107, in main\r\n    process()\r\n  File \"/opt/spark-1.2.0-bin-hadoop2.4/python/pyspark/worker.py\", line 98, in process\r\n    serializer.dump_stream(func(split_index, iterator), outfile)\r\n  File \"/opt/spark-1.2.0-bin-hadoop2.4/python/pyspark/serializers.py\", line 227, in dump_stream\r\n    vs = list(itertools.islice(iterator, batch))\r\n  File \"/net/home/ltsai/a/spark-perf.new/pyspark-tests/mllib_data.py\", line 21, in gen\r\n    rng = numpy.random.RandomState(hash(str(seed ^ index)))\r\n  File \"mtrand.pyx\", line 613, in mtrand.RandomState.__init__ (numpy/random/mtrand/mtrand.c:7402)\r\n  File \"mtrand.pyx\", line 649, in mtrand.RandomState.seed (numpy/random/mtrand/mtrand.c:7702)\r\nValueError: Seed must be between 0 and 4294967295\r\n\r\n        at org.apache.spark.api.python.PythonRDD$$anon$1.read(PythonRDD.scala:137)\r\n        at org.apache.spark.api.python.PythonRDD$$anon$1.<init>(PythonRDD.scala:174)\r\n        at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:96)\r\n        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)\r\n        at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:61)\r\n        at org.apache.spark.rdd.RDD.iterator(RDD.scala:228)\r\n        at org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply$mcV$sp(PythonRDD.scala:242)\r\n        at org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply(PythonRDD.scala:204)\r\n        at org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply(PythonRDD.scala:204)\r\n        at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1460)\r\n        at org.apache.spark.api.python.PythonRDD$WriterThread.run(PythonRDD.scala:203)\r\n```"},"comment":{"url":"https://api.github.com/repos/databricks/spark-perf/issues/comments/74723011","html_url":"https://github.com/databricks/spark-perf/issues/46#issuecomment-74723011","issue_url":"https://api.github.com/repos/databricks/spark-perf/issues/46","id":74723011,"user":{"login":"mengxr","id":829644,"avatar_url":"https://avatars.githubusercontent.com/u/829644?v=3","gravatar_id":"","url":"https://api.github.com/users/mengxr","html_url":"https://github.com/mengxr","followers_url":"https://api.github.com/users/mengxr/followers","following_url":"https://api.github.com/users/mengxr/following{/other_user}","gists_url":"https://api.github.com/users/mengxr/gists{/gist_id}","starred_url":"https://api.github.com/users/mengxr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mengxr/subscriptions","organizations_url":"https://api.github.com/users/mengxr/orgs","repos_url":"https://api.github.com/users/mengxr/repos","events_url":"https://api.github.com/users/mengxr/events{/privacy}","received_events_url":"https://api.github.com/users/mengxr/received_events","type":"User","site_admin":false},"created_at":"2015-02-17T18:32:35Z","updated_at":"2015-02-17T18:32:35Z","body":"#60 should fix this with `0xffffffff` mask."}}