{"push_id":711835355,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"61d7b533dd50bfac2162b4edcea94724bbd8fcb1","before":"74cc16dbc35e35fd5cd5542239dcb6e5e7f92d18","commits":[{"sha":"61d7b533dd50bfac2162b4edcea94724bbd8fcb1","author":{"email":"66332c93c294b372ab1bdd5acb9b89a538e2d2f2@gmail.com","name":"Yuhao Yang"},"message":"[SPARK-7514] [MLLIB] Add MinMaxScaler to feature transformation\n\njira: https://issues.apache.org/jira/browse/SPARK-7514\nAdd a popular scaling method to feature component, which is commonly known as min-max normalization or Rescaling.\n\nCore function is,\nNormalized(x) = (x - min) / (max - min) * scale + newBase\n\nwhere `newBase` and `scale` are parameters (type Double) of the `VectorTransformer`. `newBase` is the new minimum number for the features, and `scale` controls the ranges after transformation. This is a little complicated than the basic MinMax normalization, yet it provides flexibility so that users can control the range more specifically. like [0.1, 0.9] in some NN application.\n\nFor case that `max == min`, 0.5 is used as the raw value. (0.5 * scale + newBase)\nI'll add UT once the design got settled ( and this is not considered as too naive)\n\nreference:\n http://en.wikipedia.org/wiki/Feature_scaling\nhttp://stn.spotfire.com/spotfire_client_help/index.htm#norm/norm_scale_between_0_and_1.htm\n\nAuthor: Yuhao Yang <hhbyyh@gmail.com>\n\nCloses #6039 from hhbyyh/minMaxNorm and squashes the following commits:\n\nf942e9f [Yuhao Yang] add todo for metadata\n8b37bbc [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n4894dbc [Yuhao Yang] add copy\nfa2989f [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n29db415 [Yuhao Yang] add clue and minor adjustment\n5b8f7cc [Yuhao Yang] style fix\n9b133d0 [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n22f20f2 [Yuhao Yang] style change and bug fix\n747c9bb [Yuhao Yang] add ut and remove mllib version\na5ba0aa [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n585cc07 [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n1c6dcb1 [Yuhao Yang] minor change\n0f1bc80 [Yuhao Yang] add MinMaxScaler to ml\n8e7436e [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n3663165 [Yuhao Yang] Merge remote-tracking branch 'upstream/master' into minMaxNorm\n1247c27 [Yuhao Yang] some comments improvement\nd285a19 [Yuhao Yang] initial checkin for minMaxNorm","distinct":true,"url":"https://api.github.com/repos/apache/spark/commits/61d7b533dd50bfac2162b4edcea94724bbd8fcb1"}]}