{"action":"opened","number":1803,"pull_request":{"url":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803","id":47083084,"html_url":"https://github.com/spring-projects/spring-xd/pull/1803","diff_url":"https://github.com/spring-projects/spring-xd/pull/1803.diff","patch_url":"https://github.com/spring-projects/spring-xd/pull/1803.patch","issue_url":"https://api.github.com/repos/spring-projects/spring-xd/issues/1803","number":1803,"state":"open","locked":false,"title":"Add a Job DSL Parser","user":{"login":"aclement","id":226298,"avatar_url":"https://avatars.githubusercontent.com/u/226298?v=3","gravatar_id":"","url":"https://api.github.com/users/aclement","html_url":"https://github.com/aclement","followers_url":"https://api.github.com/users/aclement/followers","following_url":"https://api.github.com/users/aclement/following{/other_user}","gists_url":"https://api.github.com/users/aclement/gists{/gist_id}","starred_url":"https://api.github.com/users/aclement/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aclement/subscriptions","organizations_url":"https://api.github.com/users/aclement/orgs","repos_url":"https://api.github.com/users/aclement/repos","events_url":"https://api.github.com/users/aclement/events{/privacy}","received_events_url":"https://api.github.com/users/aclement/received_events","type":"User","site_admin":false},"body":"Related to https://jira.spring.io/browse/XD-3603\r\n\r\nI now have code blindness from staring at these changes and it is time for some feedback from you guys. This commit adds a package alongside `...dirt.stream.dsl` called `...dirt.job.dsl` and includes a `JobParser` for the Job DSL we have been talking about.\r\nNotes:\r\n-  it is not an extension of the stream parser, it is a different parser. Different tokenization, differing syntax.\r\n-  unlike the stream parser which mixed up syntactic with semantic checking, this one is pure syntax checking. There is a basic visitor pattern that could be exploited for the parsed AST where semantic validation could easily be done (i.e. is the job definition supplying the right arguments? Are the job references all references to real jobs?).\r\n- I have (in a basic way) wired it into the ToolsController for use by Flo (similarly to how the stream parser was wired in).\r\n- The stream parser used to return a structure back to the client which then got converted to a graph for Flo display. This parser can build the graph directly for return to the client. Are we ok with this? (See the Graph return value on the ToolsController request mapping)\r\n- This AST that is returned has a `toXML(batchJobId)` endpoint that produces some basic XML based on a quick exchange with Michael, this will likely need tweaking but should prove a useful start.\r\n\r\nMore work to be done (in October...):\r\n- validation\r\n- error handling (passing errors to the client).\r\n\r\nSyntax:\r\n\r\nJobs are either defined inline or reference names are used (reference names are to existing job definitions).\r\n\r\n- single job:  `foo`\r\n- single inline job definition, called 'foo': `filejdbc foo --aaa=bbb --ccc=ddd`\r\n- two jobs in a sequence: `aaa || bbb`\r\n- two jobs in parallel: `<aaa & bbb>`\r\n- a job that transitions to a different state when it exits in a certain 'state'. Here 'aaa' will jump to 'ccc' if it exits in COMPLETED state (skipping 'ddd'):\r\n  `<aaa | COMPLETED=ccc & bbb> || ddd || ccc`\r\n- transition states can be quoted if include spaces or other funky chars\r\n- multiple transition states can be specified by including multiple `| state = targetJobReference`\r\n\r\nAny feedback welcome.","created_at":"2015-10-07T21:15:01Z","updated_at":"2015-10-07T21:15:01Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803/commits","review_comments_url":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803/comments","review_comment_url":"https://api.github.com/repos/spring-projects/spring-xd/pulls/comments{/number}","comments_url":"https://api.github.com/repos/spring-projects/spring-xd/issues/1803/comments","statuses_url":"https://api.github.com/repos/spring-projects/spring-xd/statuses/11d58b54e41af44008015e25512f8947ca3d5332","head":{"label":"aclement:master","ref":"master","sha":"11d58b54e41af44008015e25512f8947ca3d5332","user":{"login":"aclement","id":226298,"avatar_url":"https://avatars.githubusercontent.com/u/226298?v=3","gravatar_id":"","url":"https://api.github.com/users/aclement","html_url":"https://github.com/aclement","followers_url":"https://api.github.com/users/aclement/followers","following_url":"https://api.github.com/users/aclement/following{/other_user}","gists_url":"https://api.github.com/users/aclement/gists{/gist_id}","starred_url":"https://api.github.com/users/aclement/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aclement/subscriptions","organizations_url":"https://api.github.com/users/aclement/orgs","repos_url":"https://api.github.com/users/aclement/repos","events_url":"https://api.github.com/users/aclement/events{/privacy}","received_events_url":"https://api.github.com/users/aclement/received_events","type":"User","site_admin":false},"repo":{"id":10486280,"name":"spring-xd","full_name":"aclement/spring-xd","owner":{"login":"aclement","id":226298,"avatar_url":"https://avatars.githubusercontent.com/u/226298?v=3","gravatar_id":"","url":"https://api.github.com/users/aclement","html_url":"https://github.com/aclement","followers_url":"https://api.github.com/users/aclement/followers","following_url":"https://api.github.com/users/aclement/following{/other_user}","gists_url":"https://api.github.com/users/aclement/gists{/gist_id}","starred_url":"https://api.github.com/users/aclement/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aclement/subscriptions","organizations_url":"https://api.github.com/users/aclement/orgs","repos_url":"https://api.github.com/users/aclement/repos","events_url":"https://api.github.com/users/aclement/events{/privacy}","received_events_url":"https://api.github.com/users/aclement/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/aclement/spring-xd","description":"Spring XD makes it easy to solve common big data problems such as data ingestion and export, real-time analytics, and batch workflow orchestration","fork":true,"url":"https://api.github.com/repos/aclement/spring-xd","forks_url":"https://api.github.com/repos/aclement/spring-xd/forks","keys_url":"https://api.github.com/repos/aclement/spring-xd/keys{/key_id}","collaborators_url":"https://api.github.com/repos/aclement/spring-xd/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/aclement/spring-xd/teams","hooks_url":"https://api.github.com/repos/aclement/spring-xd/hooks","issue_events_url":"https://api.github.com/repos/aclement/spring-xd/issues/events{/number}","events_url":"https://api.github.com/repos/aclement/spring-xd/events","assignees_url":"https://api.github.com/repos/aclement/spring-xd/assignees{/user}","branches_url":"https://api.github.com/repos/aclement/spring-xd/branches{/branch}","tags_url":"https://api.github.com/repos/aclement/spring-xd/tags","blobs_url":"https://api.github.com/repos/aclement/spring-xd/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/aclement/spring-xd/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/aclement/spring-xd/git/refs{/sha}","trees_url":"https://api.github.com/repos/aclement/spring-xd/git/trees{/sha}","statuses_url":"https://api.github.com/repos/aclement/spring-xd/statuses/{sha}","languages_url":"https://api.github.com/repos/aclement/spring-xd/languages","stargazers_url":"https://api.github.com/repos/aclement/spring-xd/stargazers","contributors_url":"https://api.github.com/repos/aclement/spring-xd/contributors","subscribers_url":"https://api.github.com/repos/aclement/spring-xd/subscribers","subscription_url":"https://api.github.com/repos/aclement/spring-xd/subscription","commits_url":"https://api.github.com/repos/aclement/spring-xd/commits{/sha}","git_commits_url":"https://api.github.com/repos/aclement/spring-xd/git/commits{/sha}","comments_url":"https://api.github.com/repos/aclement/spring-xd/comments{/number}","issue_comment_url":"https://api.github.com/repos/aclement/spring-xd/issues/comments{/number}","contents_url":"https://api.github.com/repos/aclement/spring-xd/contents/{+path}","compare_url":"https://api.github.com/repos/aclement/spring-xd/compare/{base}...{head}","merges_url":"https://api.github.com/repos/aclement/spring-xd/merges","archive_url":"https://api.github.com/repos/aclement/spring-xd/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/aclement/spring-xd/downloads","issues_url":"https://api.github.com/repos/aclement/spring-xd/issues{/number}","pulls_url":"https://api.github.com/repos/aclement/spring-xd/pulls{/number}","milestones_url":"https://api.github.com/repos/aclement/spring-xd/milestones{/number}","notifications_url":"https://api.github.com/repos/aclement/spring-xd/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/aclement/spring-xd/labels{/name}","releases_url":"https://api.github.com/repos/aclement/spring-xd/releases{/id}","created_at":"2013-06-04T18:55:33Z","updated_at":"2015-04-22T18:16:35Z","pushed_at":"2015-10-07T20:56:33Z","git_url":"git://github.com/aclement/spring-xd.git","ssh_url":"git@github.com:aclement/spring-xd.git","clone_url":"https://github.com/aclement/spring-xd.git","svn_url":"https://github.com/aclement/spring-xd","homepage":null,"size":24180,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"spring-projects:master","ref":"master","sha":"aa28ae120fb609e468f0addabbc52a7f58dfd591","user":{"login":"spring-projects","id":317776,"avatar_url":"https://avatars.githubusercontent.com/u/317776?v=3","gravatar_id":"","url":"https://api.github.com/users/spring-projects","html_url":"https://github.com/spring-projects","followers_url":"https://api.github.com/users/spring-projects/followers","following_url":"https://api.github.com/users/spring-projects/following{/other_user}","gists_url":"https://api.github.com/users/spring-projects/gists{/gist_id}","starred_url":"https://api.github.com/users/spring-projects/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spring-projects/subscriptions","organizations_url":"https://api.github.com/users/spring-projects/orgs","repos_url":"https://api.github.com/users/spring-projects/repos","events_url":"https://api.github.com/users/spring-projects/events{/privacy}","received_events_url":"https://api.github.com/users/spring-projects/received_events","type":"Organization","site_admin":false},"repo":{"id":9349347,"name":"spring-xd","full_name":"spring-projects/spring-xd","owner":{"login":"spring-projects","id":317776,"avatar_url":"https://avatars.githubusercontent.com/u/317776?v=3","gravatar_id":"","url":"https://api.github.com/users/spring-projects","html_url":"https://github.com/spring-projects","followers_url":"https://api.github.com/users/spring-projects/followers","following_url":"https://api.github.com/users/spring-projects/following{/other_user}","gists_url":"https://api.github.com/users/spring-projects/gists{/gist_id}","starred_url":"https://api.github.com/users/spring-projects/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spring-projects/subscriptions","organizations_url":"https://api.github.com/users/spring-projects/orgs","repos_url":"https://api.github.com/users/spring-projects/repos","events_url":"https://api.github.com/users/spring-projects/events{/privacy}","received_events_url":"https://api.github.com/users/spring-projects/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/spring-projects/spring-xd","description":"Spring XD makes it easy to solve common big data problems such as data ingestion and export, real-time analytics, and batch workflow orchestration","fork":false,"url":"https://api.github.com/repos/spring-projects/spring-xd","forks_url":"https://api.github.com/repos/spring-projects/spring-xd/forks","keys_url":"https://api.github.com/repos/spring-projects/spring-xd/keys{/key_id}","collaborators_url":"https://api.github.com/repos/spring-projects/spring-xd/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/spring-projects/spring-xd/teams","hooks_url":"https://api.github.com/repos/spring-projects/spring-xd/hooks","issue_events_url":"https://api.github.com/repos/spring-projects/spring-xd/issues/events{/number}","events_url":"https://api.github.com/repos/spring-projects/spring-xd/events","assignees_url":"https://api.github.com/repos/spring-projects/spring-xd/assignees{/user}","branches_url":"https://api.github.com/repos/spring-projects/spring-xd/branches{/branch}","tags_url":"https://api.github.com/repos/spring-projects/spring-xd/tags","blobs_url":"https://api.github.com/repos/spring-projects/spring-xd/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/spring-projects/spring-xd/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/spring-projects/spring-xd/git/refs{/sha}","trees_url":"https://api.github.com/repos/spring-projects/spring-xd/git/trees{/sha}","statuses_url":"https://api.github.com/repos/spring-projects/spring-xd/statuses/{sha}","languages_url":"https://api.github.com/repos/spring-projects/spring-xd/languages","stargazers_url":"https://api.github.com/repos/spring-projects/spring-xd/stargazers","contributors_url":"https://api.github.com/repos/spring-projects/spring-xd/contributors","subscribers_url":"https://api.github.com/repos/spring-projects/spring-xd/subscribers","subscription_url":"https://api.github.com/repos/spring-projects/spring-xd/subscription","commits_url":"https://api.github.com/repos/spring-projects/spring-xd/commits{/sha}","git_commits_url":"https://api.github.com/repos/spring-projects/spring-xd/git/commits{/sha}","comments_url":"https://api.github.com/repos/spring-projects/spring-xd/comments{/number}","issue_comment_url":"https://api.github.com/repos/spring-projects/spring-xd/issues/comments{/number}","contents_url":"https://api.github.com/repos/spring-projects/spring-xd/contents/{+path}","compare_url":"https://api.github.com/repos/spring-projects/spring-xd/compare/{base}...{head}","merges_url":"https://api.github.com/repos/spring-projects/spring-xd/merges","archive_url":"https://api.github.com/repos/spring-projects/spring-xd/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/spring-projects/spring-xd/downloads","issues_url":"https://api.github.com/repos/spring-projects/spring-xd/issues{/number}","pulls_url":"https://api.github.com/repos/spring-projects/spring-xd/pulls{/number}","milestones_url":"https://api.github.com/repos/spring-projects/spring-xd/milestones{/number}","notifications_url":"https://api.github.com/repos/spring-projects/spring-xd/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/spring-projects/spring-xd/labels{/name}","releases_url":"https://api.github.com/repos/spring-projects/spring-xd/releases{/id}","created_at":"2013-04-10T15:31:44Z","updated_at":"2015-10-05T17:19:32Z","pushed_at":"2015-10-07T20:20:10Z","git_url":"git://github.com/spring-projects/spring-xd.git","ssh_url":"git@github.com:spring-projects/spring-xd.git","clone_url":"https://github.com/spring-projects/spring-xd.git","svn_url":"https://github.com/spring-projects/spring-xd","homepage":"http://projects.spring.io/spring-xd","size":69002,"stargazers_count":306,"watchers_count":306,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":232,"mirror_url":null,"open_issues_count":36,"forks":232,"open_issues":36,"watchers":306,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803"},"html":{"href":"https://github.com/spring-projects/spring-xd/pull/1803"},"issue":{"href":"https://api.github.com/repos/spring-projects/spring-xd/issues/1803"},"comments":{"href":"https://api.github.com/repos/spring-projects/spring-xd/issues/1803/comments"},"review_comments":{"href":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803/comments"},"review_comment":{"href":"https://api.github.com/repos/spring-projects/spring-xd/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/spring-projects/spring-xd/pulls/1803/commits"},"statuses":{"href":"https://api.github.com/repos/spring-projects/spring-xd/statuses/11d58b54e41af44008015e25512f8947ca3d5332"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"commits":1,"additions":3796,"deletions":5,"changed_files":29}}