{"action":"created","issue":{"url":"https://api.github.com/repos/numenta/nupic.research/issues/207","labels_url":"https://api.github.com/repos/numenta/nupic.research/issues/207/labels{/name}","comments_url":"https://api.github.com/repos/numenta/nupic.research/issues/207/comments","events_url":"https://api.github.com/repos/numenta/nupic.research/issues/207/events","html_url":"https://github.com/numenta/nupic.research/issues/207","id":92224647,"number":207,"title":"TP performance on very short sequences","user":{"login":"breznak","id":2166111,"avatar_url":"https://avatars.githubusercontent.com/u/2166111?v=3","gravatar_id":"","url":"https://api.github.com/users/breznak","html_url":"https://github.com/breznak","followers_url":"https://api.github.com/users/breznak/followers","following_url":"https://api.github.com/users/breznak/following{/other_user}","gists_url":"https://api.github.com/users/breznak/gists{/gist_id}","starred_url":"https://api.github.com/users/breznak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/breznak/subscriptions","organizations_url":"https://api.github.com/users/breznak/orgs","repos_url":"https://api.github.com/users/breznak/repos","events_url":"https://api.github.com/users/breznak/events{/privacy}","received_events_url":"https://api.github.com/users/breznak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-30T23:31:44Z","updated_at":"2015-07-01T16:22:27Z","closed_at":null,"body":"I'm trying to learn many very short (size 3, predicting from 2nd element) sequences. But the performance is low. \r\n\r\nThe code is here: \r\nhttps://github.com/numenta/nupic.research/compare/numenta:master...breznak:algebraic?expand=1\r\nCould you check if the experiment is set up OK?\r\n\r\n...idea was to learn algebraic expressions, `1+1=2`\r\n\r\nResults look like this: \r\n\r\n```\r\n[2568]   (4, 3, 7) ==> acc=0.26  best=5 WRONG   confidences:  0: 0.0526,  1: 0.1088,  4: 0.0400,  5: 0.4039,  7: 0.1823,  8: 0.0394,  3: 0.0724,  6: 0.0701, \r\n[2589]   (3, 5, 8) ==> acc=0.25  best=3 WRONG   confidences:  0: 0.0952,  1: 0.1050,  5: 0.1413,  7: 0.1205,  8: 0.0874,  9: 0.0721,  10: 0.0552,  3: 0.2035, \r\n[2596]   (2, 3, 5) ==> acc=0.25  best=4 WRONG   confidences:  0: 0.1143,  1: 0.0734,  2: 0.0974,  4: 0.2259,  5: 0.0975,  7: 0.0671,  8: 0.0962,  3: 0.1587, \r\n[2608]   (2, 2, 4) ==> acc=0.25  best=4 OK      confidences:  0: 0.0776,  1: 0.0418,  2: 0.1348,  4: 0.2189,  5: 0.1735,  7: 0.0769,  6: 0.0577,  3: 0.2022, \r\n[2664]   (3, 5, 8) ==> acc=0.26  best=5 WRONG   confidences:  0: 0.1439,  1: 0.1093,  2: 0.0841,  5: 0.2009,  8: 0.1045,  9: 0.0680,  6: 0.0578,  3: 0.1028, \r\n[2711]   (0, 0, 0) ==> acc=0.26  best=0 OK      confidences:  0: 0.4520,  1: 0.1349,  2: 0.0914,  4: 0.1430,  5: 0.0700,  6: 0.0191,  3: 0.0896, \r\n[2750]   (1, 4, 5) ==> acc=0.27  best=5 OK      confidences:  0: 0.0899,  1: 0.1518,  2: 0.1397,  4: 0.1392,  5: 0.1621,  8: 0.0425,  6: 0.1325,  3: 0.1216, \r\n[2777]   (4, 1, 5) ==> acc=0.27  best=1 WRONG   confidences:  0: 0.1801,  1: 0.2078,  2: 0.0705,  4: 0.1835,  5: 0.0616,  7: 0.0022,  3: 0.1997,  6: 0.0947, \r\n[2792]   (5, 1, 6) ==> acc=0.26  best=1 WRONG   confidences:  0: 0.1470,  1: 0.2468,  2: 0.0617,  4: 0.1615,  5: 0.1048,  7: 0.0020,  3: 0.1810,  6: 0.0952, \r\n[3027]   (0, 1, 1) ==> acc=0.21  best=1 OK      confidences:  0: 0.2115,  1: 0.2204,  2: 0.0723,  4: 0.1392,  5: 0.1176,  7: 0.0034,  3: 0.1195,  6: 0.1160, \r\n[3030]   (0, 1, 1) ==> acc=0.21  best=0 WRONG   confidences:  0: 0.2207,  1: 0.1477,  2: 0.0608,  4: 0.1150,  5: 0.1217,  7: 0.0033,  3: 0.2201,  6: 0.1107, \r\n[3205]   (0, 1, 1) ==> acc=0.18  best=6 WRONG   confidences:  0: 0.1643,  1: 0.1946,  2: 0.0711,  4: 0.1420,  5: 0.0755,  7: 0.0073,  6: 0.2179,  3: 0.1273, \r\n[3248]   (0, 5, 5) ==> acc=0.16  best=3 WRONG   confidences:  1: 0.1268,  4: 0.0846,  5: 0.1033,  7: 0.0798,  9: 0.1003,  10: 0.0920,  3: 0.1341,  6: 0.1295, \r\n```"},"comment":{"url":"https://api.github.com/repos/numenta/nupic.research/issues/comments/117735463","html_url":"https://github.com/numenta/nupic.research/issues/207#issuecomment-117735463","issue_url":"https://api.github.com/repos/numenta/nupic.research/issues/207","id":117735463,"user":{"login":"subutai","id":1268932,"avatar_url":"https://avatars.githubusercontent.com/u/1268932?v=3","gravatar_id":"","url":"https://api.github.com/users/subutai","html_url":"https://github.com/subutai","followers_url":"https://api.github.com/users/subutai/followers","following_url":"https://api.github.com/users/subutai/following{/other_user}","gists_url":"https://api.github.com/users/subutai/gists{/gist_id}","starred_url":"https://api.github.com/users/subutai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/subutai/subscriptions","organizations_url":"https://api.github.com/users/subutai/orgs","repos_url":"https://api.github.com/users/subutai/repos","events_url":"https://api.github.com/users/subutai/events{/privacy}","received_events_url":"https://api.github.com/users/subutai/received_events","type":"User","site_admin":false},"created_at":"2015-07-01T16:22:27Z","updated_at":"2015-07-01T16:22:27Z","body":"> This is core of my question, I'm seeing bad performance even for trained sequences (maybe mistake in my model setup!)\r\n\r\nJust looked through your code very briefly. You might try a smaller resolution in your encoder, like 0.05. A resolution of 0.25 means that the numbers 1 and 2 will have something like 31-4=27 bits in common. The SP and TP might treat them as almost identical.\r\n\r\nSP training: how large is your training set? The SP usually requires several hundred iterations before it stabilizes. In your situation it might not be necessary to train the SP - you might try inc/dec of 0.  The TM also requires a few passes to learn high order sequences - one pass is not sufficient.\r\n\r\nBTW I don't the HTM will be able to extrapolate beyond the numbers its trained with. It's very different from the fox eat example. Even with interpolation it probably won't provide mathematically correct answers."}}