{"action":"created","issue":{"url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/20","labels_url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/20/comments","events_url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/20/events","html_url":"https://github.com/SMART-Lab/smartpy/issues/20","id":97741351,"number":20,"title":"Layer object","user":{"login":"MarcCote","id":660004,"avatar_url":"https://avatars.githubusercontent.com/u/660004?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcCote","html_url":"https://github.com/MarcCote","followers_url":"https://api.github.com/users/MarcCote/followers","following_url":"https://api.github.com/users/MarcCote/following{/other_user}","gists_url":"https://api.github.com/users/MarcCote/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcCote/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcCote/subscriptions","organizations_url":"https://api.github.com/users/MarcCote/orgs","repos_url":"https://api.github.com/users/MarcCote/repos","events_url":"https://api.github.com/users/MarcCote/events{/privacy}","received_events_url":"https://api.github.com/users/MarcCote/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-07-28T15:55:19Z","updated_at":"2015-07-28T16:53:40Z","closed_at":"2015-07-28T16:53:40Z","body":"I still think we should have `Layer` objects that we could manipulate to form the model network.\r\n\r\nThis is what I have in mind for a simple feed forward neural network with two hidden layers.\r\n```python\r\ninput_layer = Layer(size=trainset.input_size)\r\nfirst_hidden_layer = FullyConnectedLayer(size=100, activation_fct=SigmoidActivation)\r\nsecond_hidden_layer = FullyConnectedLayer(size=50, activation_fct=TanhActivation)\r\noutput_layer = FullyConnectedLayer(size=trainset.target_size, activation_fct=SigmoidActivation)\r\n\r\nnetwork = input_layer + first_hidden_layer + second_hidden_layer + output_layer\r\nmodel = NN(network, ...)\r\n```\r\n\r\nThis is an example for an RNN with two inputs.\r\n```python\r\ninput_layer = Layer(size=trainset.input_size)\r\nfirst_hidden_layer = RecurrentLayer(size=100, activation_fct=SigmoidActivation)\r\nsecond_hidden_layer = RecurrentLayer(size=50, activation_fct=TanhActivation)\r\nanother_hidden_layer = RecurrentLayer(size=50, activation_fct=ReLuActivation)\r\noutput_layer = FullyConnectedLayer(size=trainset.target_size, activation_fct=SigmoidActivation)\r\n\r\nnetwork1 = input_layer + first_hidden_layer + second_hidden_layer\r\nnetwork2 = input_layer + another_hidden_layer\r\nmerged_networks = AggregateNetworks(network1, network2, method=\"sum\")\r\nnetwork = merged_networks + output_layer\r\nmodel = RNN(network, ...)\r\n```\r\n\r\n`network.parameters` should provide a list of the parameters of every layers.\r\n\r\nMore thinking is needed to support dropout and batch normalization, maybe some sort of decorator design pattern would be useful for that.\r\n\r\nMore thinking is needed for models that have multiple outputs. @havaeimo I would love to have your input for this."},"comment":{"url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/comments/125677814","html_url":"https://github.com/SMART-Lab/smartpy/issues/20#issuecomment-125677814","issue_url":"https://api.github.com/repos/SMART-Lab/smartpy/issues/20","id":125677814,"user":{"login":"MarcCote","id":660004,"avatar_url":"https://avatars.githubusercontent.com/u/660004?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcCote","html_url":"https://github.com/MarcCote","followers_url":"https://api.github.com/users/MarcCote/followers","following_url":"https://api.github.com/users/MarcCote/following{/other_user}","gists_url":"https://api.github.com/users/MarcCote/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcCote/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcCote/subscriptions","organizations_url":"https://api.github.com/users/MarcCote/orgs","repos_url":"https://api.github.com/users/MarcCote/repos","events_url":"https://api.github.com/users/MarcCote/events{/privacy}","received_events_url":"https://api.github.com/users/MarcCote/received_events","type":"User","site_admin":false},"created_at":"2015-07-28T16:53:40Z","updated_at":"2015-07-28T16:53:40Z","body":"This has been moved over here https://github.com/SMART-Lab/models/issues/10"}}