{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1145","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1145/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1145/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1145/events","html_url":"https://github.com/fchollet/keras/issues/1145","id":120063327,"number":1145,"title":"Does the learning rate change during training?","user":{"login":"danieljf24","id":10017953,"avatar_url":"https://avatars.githubusercontent.com/u/10017953?v=3","gravatar_id":"","url":"https://api.github.com/users/danieljf24","html_url":"https://github.com/danieljf24","followers_url":"https://api.github.com/users/danieljf24/followers","following_url":"https://api.github.com/users/danieljf24/following{/other_user}","gists_url":"https://api.github.com/users/danieljf24/gists{/gist_id}","starred_url":"https://api.github.com/users/danieljf24/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danieljf24/subscriptions","organizations_url":"https://api.github.com/users/danieljf24/orgs","repos_url":"https://api.github.com/users/danieljf24/repos","events_url":"https://api.github.com/users/danieljf24/events{/privacy}","received_events_url":"https://api.github.com/users/danieljf24/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-02T23:58:20Z","updated_at":"2015-12-03T03:22:17Z","closed_at":null,"body":"rms = RMSprop()\r\nmodel.compile(loss='categorical_crossentropy', optimizer=rms)\r\n\r\nmodel.fit(X_train, Y_train,\r\n          batch_size=batch_size, nb_epoch=nb_epoch,\r\n          show_accuracy=True, verbose=2,\r\n          validation_data=(X_test, Y_test))\r\n\r\nif I used the above optimizer or any other optimizer, does the learning rate change during training?\r\nShall I change the learning rate manually?"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/161504724","html_url":"https://github.com/fchollet/keras/issues/1145#issuecomment-161504724","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1145","id":161504724,"user":{"login":"rpinsler","id":4355273,"avatar_url":"https://avatars.githubusercontent.com/u/4355273?v=3","gravatar_id":"","url":"https://api.github.com/users/rpinsler","html_url":"https://github.com/rpinsler","followers_url":"https://api.github.com/users/rpinsler/followers","following_url":"https://api.github.com/users/rpinsler/following{/other_user}","gists_url":"https://api.github.com/users/rpinsler/gists{/gist_id}","starred_url":"https://api.github.com/users/rpinsler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rpinsler/subscriptions","organizations_url":"https://api.github.com/users/rpinsler/orgs","repos_url":"https://api.github.com/users/rpinsler/repos","events_url":"https://api.github.com/users/rpinsler/events{/privacy}","received_events_url":"https://api.github.com/users/rpinsler/received_events","type":"User","site_admin":false},"created_at":"2015-12-03T03:22:17Z","updated_at":"2015-12-03T03:22:17Z","body":"Yes, the learning rate usually changes during training, depending on the optimizer you choose. As no one wants to manually change the learning rate, there are different mechanisms (so called learning rate schedules) to automatically do this. Momentum is another trick to change the learning rate.\r\n\r\nFurthermore, some optimizers such as SGD maintain a global learning rate across all parameters, whereas other approaches such as Adagrad or RMSprop effectively use per-parameter learning rates. A nice summary is given [here](http://cs231n.github.io/neural-networks-3/#update).\r\n\r\nCheck the [keras documentation](http://keras.io/optimizers/) for the default parameter values."}}