{"action":"created","issue":{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/332","labels_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/332/labels{/name}","comments_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/332/comments","events_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/332/events","html_url":"https://github.com/spark-jobserver/spark-jobserver/pull/332","id":119953876,"number":332,"title":"Improve the SparkJob API to remove code duplication","user":{"login":"reactormonk","id":49185,"avatar_url":"https://avatars.githubusercontent.com/u/49185?v=3","gravatar_id":"","url":"https://api.github.com/users/reactormonk","html_url":"https://github.com/reactormonk","followers_url":"https://api.github.com/users/reactormonk/followers","following_url":"https://api.github.com/users/reactormonk/following{/other_user}","gists_url":"https://api.github.com/users/reactormonk/gists{/gist_id}","starred_url":"https://api.github.com/users/reactormonk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reactormonk/subscriptions","organizations_url":"https://api.github.com/users/reactormonk/orgs","repos_url":"https://api.github.com/users/reactormonk/repos","events_url":"https://api.github.com/users/reactormonk/events{/privacy}","received_events_url":"https://api.github.com/users/reactormonk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-02T14:30:55Z","updated_at":"2015-12-05T23:54:21Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/pulls/332","html_url":"https://github.com/spark-jobserver/spark-jobserver/pull/332","diff_url":"https://github.com/spark-jobserver/spark-jobserver/pull/332.diff","patch_url":"https://github.com/spark-jobserver/spark-jobserver/pull/332.patch"},"body":"Upsides:\r\n- No code duplication of request parsing\r\n- Forces better coding style, no more validation all over the place\r\n\r\nDownsides:\r\n- Breaks API\r\n- `type Tmp = ...` required (can't use generics because of job classes are loaded)\r\n\r\nThings to consider:\r\n- Use `util.Try` instead of `scalaz.Validation`\r\n\r\nBugs:\r\n- somehow the Java API doesn't compile with\r\n\r\n```\r\n[error] spark-jobserver/job-server-tests/src/main/java/spark.jobserver/JavaHelloWorldJob.java:7: error: JavaHelloWorldJob is not abstract and does not override abstract method validate(Object,Config) in SparkJobBase\r\n[error] public class JavaHelloWorldJob extends JavaSparkJob {\r\n[error]        ^\r\n```"},"comment":{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/comments/162257891","html_url":"https://github.com/spark-jobserver/spark-jobserver/pull/332#issuecomment-162257891","issue_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/332","id":162257891,"user":{"login":"velvia","id":1062875,"avatar_url":"https://avatars.githubusercontent.com/u/1062875?v=3","gravatar_id":"","url":"https://api.github.com/users/velvia","html_url":"https://github.com/velvia","followers_url":"https://api.github.com/users/velvia/followers","following_url":"https://api.github.com/users/velvia/following{/other_user}","gists_url":"https://api.github.com/users/velvia/gists{/gist_id}","starred_url":"https://api.github.com/users/velvia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/velvia/subscriptions","organizations_url":"https://api.github.com/users/velvia/orgs","repos_url":"https://api.github.com/users/velvia/repos","events_url":"https://api.github.com/users/velvia/events{/privacy}","received_events_url":"https://api.github.com/users/velvia/received_events","type":"User","site_admin":false},"created_at":"2015-12-05T23:54:21Z","updated_at":"2015-12-05T23:54:21Z","body":"@reactormonk I like the idea, but I thought of a way to do this without breaking the API.   Could type members be overridden?   Maybe we can write `SparkJobBase` such that it has the Scalaz based validation and new runJob methods, but they are renamed to new methods?   Then, the existing `validate` and `runJob` methods would just call the new methods, and have a default `Tmp` (I think better name might be `JobData`) = `Config` type to maintain compatibility.  That way existing jobs still work, with a  recompilation, but people could choose to implement the new method if we want.  Meantime the examples would be modified as you did to use the new methods so folks can see the benefits.\r\n\r\n What do you think?\r\n\r\nAlso, could we still pass the job config in to `runJob` or whatever it's new method name would be (maybe `runJobWithData`?)   The reason is some jobs want to use the input system config / parameters, and there's a request to pass in the job ID and other info into a job.\r\n\r\nthanks,\r\nEvan\r\n"}}