{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/81","labels_url":"https://api.github.com/repos/fchollet/keras/issues/81/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/81/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/81/events","html_url":"https://github.com/fchollet/keras/issues/81","id":70674582,"number":81,"title":"Fix batch normalization during test time","user":{"login":"jfsantos","id":5733,"avatar_url":"https://avatars.githubusercontent.com/u/5733?v=3","gravatar_id":"","url":"https://api.github.com/users/jfsantos","html_url":"https://github.com/jfsantos","followers_url":"https://api.github.com/users/jfsantos/followers","following_url":"https://api.github.com/users/jfsantos/following{/other_user}","gists_url":"https://api.github.com/users/jfsantos/gists{/gist_id}","starred_url":"https://api.github.com/users/jfsantos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfsantos/subscriptions","organizations_url":"https://api.github.com/users/jfsantos/orgs","repos_url":"https://api.github.com/users/jfsantos/repos","events_url":"https://api.github.com/users/jfsantos/events{/privacy}","received_events_url":"https://api.github.com/users/jfsantos/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-04-24T12:17:27Z","updated_at":"2015-04-28T00:54:13Z","closed_at":null,"body":"This discussion was done in #79 but since that issue is closed I figured it would make sense to open another issue. We need to fix the batch normalization layer such that:\r\n\r\n1. It can measure the mean and variance of the batch activations of each batch it sees and store it, and\r\n2. Use that information instead of the mean and variance of the current batch during testing.\r\n\r\nRegarding 1, I think maybe it's not good to measure the activation statistics during training because they will be changing over time. Maybe a safer way is to wait until training is over, and then measure these over a single epoch, with all network parameters static."},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/96861179","html_url":"https://github.com/fchollet/keras/issues/81#issuecomment-96861179","issue_url":"https://api.github.com/repos/fchollet/keras/issues/81","id":96861179,"user":{"login":"jfsantos","id":5733,"avatar_url":"https://avatars.githubusercontent.com/u/5733?v=3","gravatar_id":"","url":"https://api.github.com/users/jfsantos","html_url":"https://github.com/jfsantos","followers_url":"https://api.github.com/users/jfsantos/followers","following_url":"https://api.github.com/users/jfsantos/following{/other_user}","gists_url":"https://api.github.com/users/jfsantos/gists{/gist_id}","starred_url":"https://api.github.com/users/jfsantos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfsantos/subscriptions","organizations_url":"https://api.github.com/users/jfsantos/orgs","repos_url":"https://api.github.com/users/jfsantos/repos","events_url":"https://api.github.com/users/jfsantos/events{/privacy}","received_events_url":"https://api.github.com/users/jfsantos/received_events","type":"User","site_admin":false},"created_at":"2015-04-28T00:54:13Z","updated_at":"2015-04-28T00:54:13Z","body":"We could try something like the online/parallel variance calculation methods described [here](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm). I never used them and this is the first time I see anything about this, but it may be something interesting to play with."}}