{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/3222","labels_url":"https://api.github.com/repos/apache/spark/issues/3222/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/3222/comments","events_url":"https://api.github.com/repos/apache/spark/issues/3222/events","html_url":"https://github.com/apache/spark/pull/3222","id":48488624,"number":3222,"title":"[WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, ANN, BDN algorithm to MLlib","user":{"login":"witgo","id":302879,"avatar_url":"https://avatars.githubusercontent.com/u/302879?v=3","gravatar_id":"","url":"https://api.github.com/users/witgo","html_url":"https://github.com/witgo","followers_url":"https://api.github.com/users/witgo/followers","following_url":"https://api.github.com/users/witgo/following{/other_user}","gists_url":"https://api.github.com/users/witgo/gists{/gist_id}","starred_url":"https://api.github.com/users/witgo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/witgo/subscriptions","organizations_url":"https://api.github.com/users/witgo/orgs","repos_url":"https://api.github.com/users/witgo/repos","events_url":"https://api.github.com/users/witgo/events{/privacy}","received_events_url":"https://api.github.com/users/witgo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":92,"created_at":"2014-11-12T09:16:24Z","updated_at":"2015-01-28T06:32:08Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/3222","html_url":"https://github.com/apache/spark/pull/3222","diff_url":"https://github.com/apache/spark/pull/3222.diff","patch_url":"https://github.com/apache/spark/pull/3222.patch"},"body":"### Activation function\r\n- [X] softmax\r\n- [X] sigmoid\r\n- [X] tanh\r\n- [X] softplus\r\n- [X] ReLu\r\n- [X] Noisy ReLU\r\n \r\n### Gradient descent\r\n\r\n- [X] AdaDelta \r\n- [X] AdaGrad \r\n- [X] L-BFGS \r\n- [X] Momentum\r\n\r\n### Regularization method\r\n\r\n- [X] Dropout\r\n- [X] L2\r\n \r\n### Experimental Results\r\n\r\n#### mnist dataset\r\n\r\n##### DBN\r\n\r\n   network structure: 784 * 300 * 300 * 500 * 10; Gradient descent: AdaGrad;learning Rate: 0.05; Weight cost 0.0; Dropout rate: 0.5, miniBatch: 300; Training data: 60000; Test data: 10000;    \r\n\r\n\t    Iterations | Test error\r\n\t\t------------ | -------------\r\n\t\t6000 | 1.96 %\r\n        20000 |  1.70%\r\n    \r\n    \r\n### REFERENCES\r\n*  J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Ng, \"[Large scale distributed deep networks](http://www.cs.toronto.edu/~ranzato/publications/DistBeliefNIPS2012_withAppendix.pdf) \" in NIPS, 2012.\r\n\r\n* Matthew D. Zeiler,  \"[ADADELTA: An Adaptive Learning Rate Method](http://arxiv.org/abs/1212.5701)\", abs/1212.5701, 2012.\r\n\r\n* Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. \"[Improving neural networks by preventing co-adaptation of feature detectors](http://arxiv.org/abs/1207.0580)\", CoRR, abs/1207.0580, 2012.\r\n\r\n* N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, \"[Dropout: A simple way to prevent neural networks from overfitting](http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf)\", Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/71787790","html_url":"https://github.com/apache/spark/pull/3222#issuecomment-71787790","issue_url":"https://api.github.com/repos/apache/spark/issues/3222","id":71787790,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-01-28T06:32:07Z","updated_at":"2015-01-28T06:32:07Z","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26207/\nTest PASSed."}}