{"action":"created","issue":{"url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/17","labels_url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/17/labels{/name}","comments_url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/17/comments","events_url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/17/events","html_url":"https://github.com/sequenceiq/docker-spark/issues/17","id":62243487,"number":17,"title":"stateful python streaming examples fail","user":{"login":"mhowsden","id":353359,"avatar_url":"https://avatars.githubusercontent.com/u/353359?v=3","gravatar_id":"","url":"https://api.github.com/users/mhowsden","html_url":"https://github.com/mhowsden","followers_url":"https://api.github.com/users/mhowsden/followers","following_url":"https://api.github.com/users/mhowsden/following{/other_user}","gists_url":"https://api.github.com/users/mhowsden/gists{/gist_id}","starred_url":"https://api.github.com/users/mhowsden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhowsden/subscriptions","organizations_url":"https://api.github.com/users/mhowsden/orgs","repos_url":"https://api.github.com/users/mhowsden/repos","events_url":"https://api.github.com/users/mhowsden/events{/privacy}","received_events_url":"https://api.github.com/users/mhowsden/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-03-17T00:19:37Z","updated_at":"2015-08-19T05:57:44Z","closed_at":"2015-08-03T16:58:24Z","body":"Thanks so much for doing this, this docker has come in handy getting me going with spark.\r\n\r\nI tried the streaming examples with \r\n\r\n    spark-submit /usr/local/spark/examples/src/main/python/streaming/network_wordcount.py 172.17.42.1 9999 2> out.txt\r\n\r\nAnd it works great but when I run\r\n\r\n    spark-submit /usr/local/spark/examples/src/main/python/streaming/stateful_network_wordcount.py 172.17.42.1 9999 2> out.txt\r\n\r\nI don't get any results and the errors I get are\r\n\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job 1426550835000 ms.0\r\n    py4j.Py4JException: An exception was raised by the Python Proxy. Return Message: null\r\n        at py4j.Protocol.getReturnValue(Protocol.java:417)\r\n    --\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job  1426550836000 ms.0\r\n    py4j.Py4JException: An exception was raised by the Python Proxy. Return Message: null\r\n        at py4j.Protocol.getReturnValue(Protocol.java:417)\r\n    --\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job 1426550837000 ms.0\r\n    py4j.Py4JException: An exception was raised by the Python Proxy. Return Message: null\r\n        at py4j.Protocol.getReturnValue(Protocol.java:417)\r\n    --\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job 1426550838000 ms.0\r\n    py4j.Py4JException: Error while obtaining a new communication channel\r\n        at py4j.CallbackClient.getConnectionLock(CallbackClient.java:155)\r\n    --\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job 1426550839000 ms.0\r\n    py4j.Py4JException: Error while obtaining a new communication channel\r\n        at py4j.CallbackClient.getConnectionLock(CallbackClient.java:155)\r\n    --\r\n    15/03/16 20:09:23 ERROR scheduler.JobScheduler: Error running job streaming job 1426550840000 ms.0\r\n    py4j.Py4JException: Error while obtaining a new communication channel\r\n        at py4j.CallbackClient.getConnectionLock(CallbackClient.java:155)\r\n\r\nApologies if this is just a misunderstanding of how I need to run this in the context of docker or a simple flag I need to add to the spark-submit command.  I'm new to spark (and docker) and I'm unclear about what's failing.  The recoverable_network_wordcount.py failed as well."},"comment":{"url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/comments/132454997","html_url":"https://github.com/sequenceiq/docker-spark/issues/17#issuecomment-132454997","issue_url":"https://api.github.com/repos/sequenceiq/docker-spark/issues/17","id":132454997,"user":{"login":"vishnevski","id":3044657,"avatar_url":"https://avatars.githubusercontent.com/u/3044657?v=3","gravatar_id":"","url":"https://api.github.com/users/vishnevski","html_url":"https://github.com/vishnevski","followers_url":"https://api.github.com/users/vishnevski/followers","following_url":"https://api.github.com/users/vishnevski/following{/other_user}","gists_url":"https://api.github.com/users/vishnevski/gists{/gist_id}","starred_url":"https://api.github.com/users/vishnevski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vishnevski/subscriptions","organizations_url":"https://api.github.com/users/vishnevski/orgs","repos_url":"https://api.github.com/users/vishnevski/repos","events_url":"https://api.github.com/users/vishnevski/events{/privacy}","received_events_url":"https://api.github.com/users/vishnevski/received_events","type":"User","site_admin":false},"created_at":"2015-08-19T05:57:44Z","updated_at":"2015-08-19T05:57:44Z","body":"Spark streaming needs at least two threads on the worker/slave side.\r\nIt works like: ./bin/spark-submit --master local[2]  examples/src/main/python/streaming/network_wordcount.py 127.0.0.1 9999 \r\nFound here http://comments.gmane.org/gmane.comp.lang.scala.spark.user/14543\r\n"}}