{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/5820","labels_url":"https://api.github.com/repos/apache/spark/issues/5820/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/5820/comments","events_url":"https://api.github.com/repos/apache/spark/issues/5820/events","html_url":"https://github.com/apache/spark/pull/5820","id":72277736,"number":5820,"title":"[SPARK-5956][MLLIB] Pipeline components should be copyable.","user":{"login":"mengxr","id":829644,"avatar_url":"https://avatars.githubusercontent.com/u/829644?v=3","gravatar_id":"","url":"https://api.github.com/users/mengxr","html_url":"https://github.com/mengxr","followers_url":"https://api.github.com/users/mengxr/followers","following_url":"https://api.github.com/users/mengxr/following{/other_user}","gists_url":"https://api.github.com/users/mengxr/gists{/gist_id}","starred_url":"https://api.github.com/users/mengxr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mengxr/subscriptions","organizations_url":"https://api.github.com/users/mengxr/orgs","repos_url":"https://api.github.com/users/mengxr/repos","events_url":"https://api.github.com/users/mengxr/events{/privacy}","received_events_url":"https://api.github.com/users/mengxr/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-04-30T20:49:38Z","updated_at":"2015-04-30T22:00:08Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/5820","html_url":"https://github.com/apache/spark/pull/5820","diff_url":"https://github.com/apache/spark/pull/5820.diff","patch_url":"https://github.com/apache/spark/pull/5820.patch"},"body":"This PR added `copy(extra: ParamMap): Params` to `Params`, which makes a copy of the current instance with a randomly generated uid and some extra param values. With this change, we only need to implement `fit` and `transform` without extra param values given the default implementation of `fit(dataset, extra)`:\r\n\r\n~~~scala\r\ndef fit(dataset: DataFrame, extra: ParamMap): Model = {\r\n  copy(extra).fit(dataset)\r\n}\r\n~~~\r\n\r\nInside `fit` and `transform`, since only the embedded values are used, I added `$` as an alias for `getOrDefault` to make the code easier to read. For example, in `LinearRegression.fit` we have:\r\n\r\n~~~scala\r\nval effectiveRegParam = $(regParam) / yStd\r\nval effectiveL1RegParam = $(elasticNetParam) * effectiveRegParam\r\nval effectiveL2RegParam = (1.0 - $(elasticNetParam)) * effectiveRegParam\r\n~~~\r\n\r\nMeta-algorithm like `Pipeline` implements its own `copy(extra)`. So the fitted pipeline model stored all copied stages (no matter whether it is a transformer or a model).\r\n\r\nOther changes:\r\n* `Params$.inheritValues` is moved to `Params!.copyValues` and returns the target instance.\r\n* `fittingParamMap` was removed because the `parent` carries this information.\r\n* `validate` was renamed to `validateParams` to be more precise.\r\n\r\nTODOs:\r\n* [ ] add tests for newly added methods\r\n* [ ] update documentation\r\n\r\n@jkbradley"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/97983230","html_url":"https://github.com/apache/spark/pull/5820#issuecomment-97983230","issue_url":"https://api.github.com/repos/apache/spark/issues/5820","id":97983230,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-04-30T22:00:07Z","updated_at":"2015-04-30T22:00:07Z","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31451/\nTest FAILed."}}