{"action":"created","issue":{"url":"https://api.github.com/repos/piskvorky/gensim/issues/298","labels_url":"https://api.github.com/repos/piskvorky/gensim/issues/298/labels{/name}","comments_url":"https://api.github.com/repos/piskvorky/gensim/issues/298/comments","events_url":"https://api.github.com/repos/piskvorky/gensim/issues/298/events","html_url":"https://github.com/piskvorky/gensim/issues/298","id":59178557,"number":298,"title":"LDA model Incorrect treatment of damping factor when update() has multiple passes","user":{"login":"wbogorad","id":7495432,"avatar_url":"https://avatars.githubusercontent.com/u/7495432?v=3","gravatar_id":"","url":"https://api.github.com/users/wbogorad","html_url":"https://github.com/wbogorad","followers_url":"https://api.github.com/users/wbogorad/followers","following_url":"https://api.github.com/users/wbogorad/following{/other_user}","gists_url":"https://api.github.com/users/wbogorad/gists{/gist_id}","starred_url":"https://api.github.com/users/wbogorad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wbogorad/subscriptions","organizations_url":"https://api.github.com/users/wbogorad/orgs","repos_url":"https://api.github.com/users/wbogorad/repos","events_url":"https://api.github.com/users/wbogorad/events{/privacy}","received_events_url":"https://api.github.com/users/wbogorad/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-02-27T02:06:52Z","updated_at":"2015-02-28T21:52:18Z","closed_at":null,"body":"Radim,\r\n\r\nI think that rhot gets changed too frequently in ldamodel.update() than it should in online mode.\r\nrhot is only supposed to damp new chunk of documents with time according to Hoffman's Algorithm 2.\r\n\r\nHowever, in case of multiple passes it also damps the the updates progressively with each pass.\r\nI am not sure that this is correct. \r\nThis is because do_mstep() is called during each pass and it increases self.num_updates variable that drives rhot\r\nAs another consequence new chunks get suppressed more than they should.\r\n\r\nBy the way, I appreciate that you introduced multiple passes, Hoffman does not have them and in my experiments they do help achieving more robust convergence.\r\n \r\n What do you think?"},"comment":{"url":"https://api.github.com/repos/piskvorky/gensim/issues/comments/76548617","html_url":"https://github.com/piskvorky/gensim/issues/298#issuecomment-76548617","issue_url":"https://api.github.com/repos/piskvorky/gensim/issues/298","id":76548617,"user":{"login":"cscorley","id":112752,"avatar_url":"https://avatars.githubusercontent.com/u/112752?v=3","gravatar_id":"","url":"https://api.github.com/users/cscorley","html_url":"https://github.com/cscorley","followers_url":"https://api.github.com/users/cscorley/followers","following_url":"https://api.github.com/users/cscorley/following{/other_user}","gists_url":"https://api.github.com/users/cscorley/gists{/gist_id}","starred_url":"https://api.github.com/users/cscorley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cscorley/subscriptions","organizations_url":"https://api.github.com/users/cscorley/orgs","repos_url":"https://api.github.com/users/cscorley/repos","events_url":"https://api.github.com/users/cscorley/events{/privacy}","received_events_url":"https://api.github.com/users/cscorley/received_events","type":"User","site_admin":false},"created_at":"2015-02-28T21:52:18Z","updated_at":"2015-02-28T21:52:18Z","body":"Let me see if I follow you correctly:\r\n\r\nPasses effectively repeats the corpus given to `update()`, increasing `t` (=chunks/minibatches seen) because each chunk is seen again with each pass. For the first pass, everything is as it should be (in Algo 2). After the first pass, `t` is not reset; rather, it begins as if it has begun to seen more corpus (even though they are the same chunks). \r\n\r\nThis seems desirable to me; each pass through the corpus is damped more and more. \r\n\r\nHowever, you are right about using `update()` on new corpora/chunks. E.g., if we use a single pass on corpus `C_a` (=10 chunks), then `t=10`. Later giving `update()` another corpus `C_b` (=7 chunks), then `t=17`. When multiple passes are introduced, say 5 passes, then after updating with C_a, `t=50`. This is bad news for `C_b`, it seems, as it will not start with `t=10`, but `t=50`. Yikes!\r\n\r\nWhat would be a better way to approach this? Ensure that after a corpus has been through all of it's passes that `t` is set back to what it was after the first pass? E.g., following the above example, `C_a` begins with `t=0`, increases it to 50, but leaves it at 10. That way, `C_b` begins with `t=10`, increases it to 45, but leaves it at 17. Does it make sense to do that?"}}