{"action":"created","issue":{"url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/19","labels_url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/19/labels{/name}","comments_url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/19/comments","events_url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/19/events","html_url":"https://github.com/ctn-waterloo/modelling_ideas/issues/19","id":74673657,"number":19,"title":"Operant conditioning and the cerebellum","user":{"login":"tcstewar","id":2380772,"avatar_url":"https://avatars.githubusercontent.com/u/2380772?v=3","gravatar_id":"","url":"https://api.github.com/users/tcstewar","html_url":"https://github.com/tcstewar","followers_url":"https://api.github.com/users/tcstewar/followers","following_url":"https://api.github.com/users/tcstewar/following{/other_user}","gists_url":"https://api.github.com/users/tcstewar/gists{/gist_id}","starred_url":"https://api.github.com/users/tcstewar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcstewar/subscriptions","organizations_url":"https://api.github.com/users/tcstewar/orgs","repos_url":"https://api.github.com/users/tcstewar/repos","events_url":"https://api.github.com/users/tcstewar/events{/privacy}","received_events_url":"https://api.github.com/users/tcstewar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-05-09T13:30:36Z","updated_at":"2015-05-09T14:38:07Z","closed_at":null,"body":"Here's a quick implementation of operant conditioning that's pretty much the same as the model in DAC and other places.  The idea is that the cerebellum is a forward model that takes all the stimuli from everywhere in the brain (CS) and uses that to predict whatever automatic actions are currently being triggered by some hard-wired reflexes (US).  \r\n\r\nThis is basically just supervised learning, so let's use PES\r\n\r\n```python\r\nimport nengo\r\n\r\nD = 3\r\nN = D*50\r\n\r\nmodel = nengo.Network(label=\"My Network\")\r\nwith model:\r\n    \r\n    def us_stim(t):\r\n        t = t % 3\r\n        if 0.9 < t< 1: return [1, 0, 0]\r\n        if 1.9 < t< 2: return [0, 1, 0]\r\n        if 2.9 < t< 3: return [0, 0, 1]\r\n        return [0, 0, 0]\r\n    us_stim = nengo.Node(us_stim)\r\n\r\n    def cs_stim(t):\r\n        t = t % 3\r\n        if 0.7 < t< 1: return [0.7, 0, 0.7]\r\n        if 1.7 < t< 2: return [0.6, 0.6, 0.6]\r\n        if 2.7 < t< 3: return [0, 1, 0]\r\n        return [0, 0, 0]\r\n    cs_stim = nengo.Node(cs_stim)\r\n    \r\n    us = nengo.Ensemble(N, D)\r\n    cs = nengo.Ensemble(N*2, D*2)\r\n    \r\n    nengo.Connection(us_stim, us[:D])\r\n    nengo.Connection(cs_stim, cs[:D])\r\n    nengo.Connection(cs[:D], cs[D:], synapse=0.2)\r\n    \r\n    \r\n    act = nengo.Ensemble(N, D)\r\n    \r\n    nengo.Connection(us, act)\r\n    \r\n    predict = nengo.Ensemble(N, D)\r\n    \r\n    learn_conn = nengo.Connection(cs, predict, function=lambda x: [0]*D)\r\n    \r\n    error = nengo.Ensemble(N, D)\r\n    nengo.Connection(act, error)\r\n    nengo.Connection(predict, error, transform=-1, synapse=0.1)\r\n    \r\n    error_conn = nengo.Connection(error, predict, modulatory=True)\r\n    \r\n    learn_conn.learning_rule_type = nengo.PES(error_conn, \r\n                                              learning_rate=3e-6)\r\n    \r\n    motor = nengo.Ensemble(N, D)\r\n    nengo.Connection(act, motor)\r\n    nengo.Connection(predict, motor)\r\n```"},"comment":{"url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/comments/100497942","html_url":"https://github.com/ctn-waterloo/modelling_ideas/issues/19#issuecomment-100497942","issue_url":"https://api.github.com/repos/ctn-waterloo/modelling_ideas/issues/19","id":100497942,"user":{"login":"tcstewar","id":2380772,"avatar_url":"https://avatars.githubusercontent.com/u/2380772?v=3","gravatar_id":"","url":"https://api.github.com/users/tcstewar","html_url":"https://github.com/tcstewar","followers_url":"https://api.github.com/users/tcstewar/followers","following_url":"https://api.github.com/users/tcstewar/following{/other_user}","gists_url":"https://api.github.com/users/tcstewar/gists{/gist_id}","starred_url":"https://api.github.com/users/tcstewar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcstewar/subscriptions","organizations_url":"https://api.github.com/users/tcstewar/orgs","repos_url":"https://api.github.com/users/tcstewar/repos","events_url":"https://api.github.com/users/tcstewar/events{/privacy}","received_events_url":"https://api.github.com/users/tcstewar/received_events","type":"User","site_admin":false},"created_at":"2015-05-09T14:38:07Z","updated_at":"2015-05-09T14:38:07Z","body":"(and perhaps the cs population as a sparsified version...  )"}}