{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6967","labels_url":"https://api.github.com/repos/apache/spark/issues/6967/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6967/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6967/events","html_url":"https://github.com/apache/spark/pull/6967","id":90536067,"number":6967,"title":"[SPARK-8583] [SPARK-5482] [BUILD] Refactor python/run-tests to integrate with dev/run-tests module system","user":{"login":"JoshRosen","id":50748,"avatar_url":"https://avatars.githubusercontent.com/u/50748?v=3","gravatar_id":"","url":"https://api.github.com/users/JoshRosen","html_url":"https://github.com/JoshRosen","followers_url":"https://api.github.com/users/JoshRosen/followers","following_url":"https://api.github.com/users/JoshRosen/following{/other_user}","gists_url":"https://api.github.com/users/JoshRosen/gists{/gist_id}","starred_url":"https://api.github.com/users/JoshRosen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JoshRosen/subscriptions","organizations_url":"https://api.github.com/users/JoshRosen/orgs","repos_url":"https://api.github.com/users/JoshRosen/repos","events_url":"https://api.github.com/users/JoshRosen/events{/privacy}","received_events_url":"https://api.github.com/users/JoshRosen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":65,"created_at":"2015-06-24T01:34:14Z","updated_at":"2015-06-26T08:12:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6967","html_url":"https://github.com/apache/spark/pull/6967","diff_url":"https://github.com/apache/spark/pull/6967.diff","patch_url":"https://github.com/apache/spark/pull/6967.patch"},"body":"This patch refactors the `python/run-tests` script:\r\n\r\n- It's now written in Python instead of Bash.\r\n- The descriptions of the tests to run are now stored in `dev/run-tests`'s modules.  This allows the pull request builder to skip Python tests suites that were not affected by the pull request's changes.  For example, we can now skip the PySpark Streaming test cases when only SQL files are changed.\r\n- `python/run-tests` now supports command-line flags to make it easier to run individual test suites (this addresses SPARK-5482):\r\n\r\n  ```\r\nUsage: run-tests [options]\r\n\r\nOptions:\r\n  -h, --help            show this help message and exit\r\n  --python-executables=PYTHON_EXECUTABLES\r\n                        A comma-separated list of Python executables to test\r\n                        against (default: python2.6,python3.4,pypy)\r\n  --modules=MODULES     A comma-separated list of Python modules to test\r\n                        (default: pyspark-core,pyspark-ml,pyspark-mllib\r\n                        ,pyspark-sql,pyspark-streaming)\r\n   ```\r\n- `dev/run-tests` has been split into multiple files: the module definitions and test utility functions are now stored inside of a `dev/sparktestsupport` Python module, allowing them to be re-used from the Python test runner script."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/115568766","html_url":"https://github.com/apache/spark/pull/6967#issuecomment-115568766","issue_url":"https://api.github.com/repos/apache/spark/issues/6967","id":115568766,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-06-26T08:12:16Z","updated_at":"2015-06-26T08:12:16Z","body":" Merged build triggered."}}