{"action":"created","issue":{"url":"https://api.github.com/repos/casperkaae/parmesan/issues/22","labels_url":"https://api.github.com/repos/casperkaae/parmesan/issues/22/labels{/name}","comments_url":"https://api.github.com/repos/casperkaae/parmesan/issues/22/comments","events_url":"https://api.github.com/repos/casperkaae/parmesan/issues/22/events","html_url":"https://github.com/casperkaae/parmesan/issues/22","id":114576338,"number":22,"title":"Reproduce results from sec. 6.1 in \"Variational inference using normalizing flows\"","user":{"login":"casperkaae","id":3503745,"avatar_url":"https://avatars.githubusercontent.com/u/3503745?v=3","gravatar_id":"","url":"https://api.github.com/users/casperkaae","html_url":"https://github.com/casperkaae","followers_url":"https://api.github.com/users/casperkaae/followers","following_url":"https://api.github.com/users/casperkaae/following{/other_user}","gists_url":"https://api.github.com/users/casperkaae/gists{/gist_id}","starred_url":"https://api.github.com/users/casperkaae/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/casperkaae/subscriptions","organizations_url":"https://api.github.com/users/casperkaae/orgs","repos_url":"https://api.github.com/users/casperkaae/repos","events_url":"https://api.github.com/users/casperkaae/events{/privacy}","received_events_url":"https://api.github.com/users/casperkaae/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-11-02T11:27:04Z","updated_at":"2015-11-09T12:56:22Z","closed_at":null,"body":"As discussed in #21 it would be nice to reproduce the results from sec. 6.1 in the \"Variational inference using normalizing flows\" paper by Rezende et al.\r\n\r\nI would guess the approach is:\r\n- sample input: x = q0 = N(z; mu, sigma^2 I)\r\n- run through mapping and normalizing flow to get p(U_hat)\r\n- update params using squared error between U and U_hat\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/casperkaae/parmesan/issues/comments/155055283","html_url":"https://github.com/casperkaae/parmesan/issues/22#issuecomment-155055283","issue_url":"https://api.github.com/repos/casperkaae/parmesan/issues/22","id":155055283,"user":{"login":"wuaalb","id":8669162,"avatar_url":"https://avatars.githubusercontent.com/u/8669162?v=3","gravatar_id":"","url":"https://api.github.com/users/wuaalb","html_url":"https://github.com/wuaalb","followers_url":"https://api.github.com/users/wuaalb/followers","following_url":"https://api.github.com/users/wuaalb/following{/other_user}","gists_url":"https://api.github.com/users/wuaalb/gists{/gist_id}","starred_url":"https://api.github.com/users/wuaalb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wuaalb/subscriptions","organizations_url":"https://api.github.com/users/wuaalb/orgs","repos_url":"https://api.github.com/users/wuaalb/repos","events_url":"https://api.github.com/users/wuaalb/events{/privacy}","received_events_url":"https://api.github.com/users/wuaalb/received_events","type":"User","site_admin":false},"created_at":"2015-11-09T12:56:22Z","updated_at":"2015-11-09T12:56:22Z","body":"Sounds interesting.. but I'm not completely sure what you mean; could you post the exact change? What's \"the loss jf\"?\r\n\r\nI ran some more experiments; and fixed the 3rd plot (so 3rd and 4th plot should be identical).\r\nWith `nflows=8` I get some thing similar to you\r\n![_nf2_nflows 8_winit normal 0](https://cloud.githubusercontent.com/assets/8669162/11034015/5659596e-86e8-11e5-8676-27157a2c0c00.png)\r\n\r\nHowever, if I initialize with `w=lasagne.init.Normal(mean=[1.0, 0.0])`, it is able to split the distribution into a bimodal one.\r\n![_nf2_nflows 8_winit 1 0](https://cloud.githubusercontent.com/assets/8669162/11034058/bf3d7546-86e8-11e5-9dc3-75695956eba8.png)\r\n\r\nSo, I think this at least means that `NormalizingPlanarFlowLayer` outputs the correct `f(z)` and `log det |J|`. So *if* it has a problem, it is probably in how the constraints are applied or in the initialization of the parameters; or (perhaps more likely) there's something wrong with the loss or optimization."}}