{"action":"created","issue":{"url":"https://api.github.com/repos/bartvm/blocks/issues/280","labels_url":"https://api.github.com/repos/bartvm/blocks/issues/280/labels{/name}","comments_url":"https://api.github.com/repos/bartvm/blocks/issues/280/comments","events_url":"https://api.github.com/repos/bartvm/blocks/issues/280/events","html_url":"https://github.com/bartvm/blocks/issues/280","id":57650659,"number":280,"title":"Dropout","user":{"login":"harmdevries89","id":8825215,"avatar_url":"https://avatars.githubusercontent.com/u/8825215?v=3","gravatar_id":"","url":"https://api.github.com/users/harmdevries89","html_url":"https://github.com/harmdevries89","followers_url":"https://api.github.com/users/harmdevries89/followers","following_url":"https://api.github.com/users/harmdevries89/following{/other_user}","gists_url":"https://api.github.com/users/harmdevries89/gists{/gist_id}","starred_url":"https://api.github.com/users/harmdevries89/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harmdevries89/subscriptions","organizations_url":"https://api.github.com/users/harmdevries89/orgs","repos_url":"https://api.github.com/users/harmdevries89/repos","events_url":"https://api.github.com/users/harmdevries89/events{/privacy}","received_events_url":"https://api.github.com/users/harmdevries89/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bartvm/blocks/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/bartvm/blocks/labels/major","name":"major","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-02-13T20:35:48Z","updated_at":"2015-02-16T15:18:15Z","closed_at":null,"body":"I would like to have your thoughts about the best way to implement Dropout in blocks. In my opinion there are two options:\r\n- Suggested by @bartvm: After construction of the model, manipulate the theano graph. One problem I see is that hidden units should be dropped, which, as far as I know, are not directly accessible with the Computational Graph. We can probably overcome this by cleverly masking weights and biases. \r\n- Make dropout part of the model architecture. We then need to have bricks that have a different apply methods at training time and test time. \r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/bartvm/blocks/issues/comments/74523203","html_url":"https://github.com/bartvm/blocks/issues/280#issuecomment-74523203","issue_url":"https://api.github.com/repos/bartvm/blocks/issues/280","id":74523203,"user":{"login":"bartvm","id":2299595,"avatar_url":"https://avatars.githubusercontent.com/u/2299595?v=3","gravatar_id":"","url":"https://api.github.com/users/bartvm","html_url":"https://github.com/bartvm","followers_url":"https://api.github.com/users/bartvm/followers","following_url":"https://api.github.com/users/bartvm/following{/other_user}","gists_url":"https://api.github.com/users/bartvm/gists{/gist_id}","starred_url":"https://api.github.com/users/bartvm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bartvm/subscriptions","organizations_url":"https://api.github.com/users/bartvm/orgs","repos_url":"https://api.github.com/users/bartvm/repos","events_url":"https://api.github.com/users/bartvm/events{/privacy}","received_events_url":"https://api.github.com/users/bartvm/received_events","type":"User","site_admin":false},"created_at":"2015-02-16T15:18:15Z","updated_at":"2015-02-16T15:18:15Z","body":"I don't like your defaults as much though. I think that if a user noises his weights, he should be able to retrieve them by default, not needing to know the meaning of an obscure `follow_links` argument. I'd like these replacements to be more transparent and easily accessible to users.\r\n\r\nIt's true that you can retrieve the original roles from the replaced variable, but creating the roles dynamically makes it possible for the user to interact with the roles directly more easily and do things like: `WEIGHTS in var.tag.roles` or `any(isinstance(_, PARAMETER) for _ in var.tag.roles)` to check whether a variable they have are weights or not, even if they are noised. Otherwise a user would have to immediately resort to writing stuff like\r\n```python\r\ndef recursive_role_checker(var)\r\n    if hasattr(var.tag, 'replaces'):\r\n        recursive_role_checker(var.tag.replaces)\r\n    else:\r\n        return any(isinstance(_, PARAMETER) for _ in var.tag.roles)\r\n```\r\nGetting the non-noised variables should be solved by adding a `not_roles` keyword argument, which we should support anyway for cases like \"give me all parameters except biases\", as in:\r\n```python\r\n>>> VariableFilter(roles=[WEIGHTS], not_roles=[NOISED])(cg.variables)\r\n[W1_noised, W2]\r\n```\r\nI think that being able to pass `not_roles` and being able to still interact with a noise weight matrix as if it is a weight matrix is more user-friendly, even if it adds a tiny bit of redundancy by wrapping the original role into a new one."}}