{"action":"created","issue":{"url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/7","labels_url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/7/labels{/name}","comments_url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/7/comments","events_url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/7/events","html_url":"https://github.com/DiffSharp/DiffSharp/issues/7","id":94954558,"number":7,"title":"reversePush results in Stack overflow for larger than small networks","user":{"login":"jragonslayer","id":10254003,"avatar_url":"https://avatars.githubusercontent.com/u/10254003?v=3","gravatar_id":"","url":"https://api.github.com/users/jragonslayer","html_url":"https://github.com/jragonslayer","followers_url":"https://api.github.com/users/jragonslayer/followers","following_url":"https://api.github.com/users/jragonslayer/following{/other_user}","gists_url":"https://api.github.com/users/jragonslayer/gists{/gist_id}","starred_url":"https://api.github.com/users/jragonslayer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jragonslayer/subscriptions","organizations_url":"https://api.github.com/users/jragonslayer/orgs","repos_url":"https://api.github.com/users/jragonslayer/repos","events_url":"https://api.github.com/users/jragonslayer/events{/privacy}","received_events_url":"https://api.github.com/users/jragonslayer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-07-14T14:17:21Z","updated_at":"2015-07-14T14:55:09Z","closed_at":null,"body":"Since reversePush is not tail recursive, it results in a stack overflow for even simple networks when you push in a lot of data/use a bunch of memory. Here's the code I used to produce this (backprop is modified to allow a linear output layer and different activation functions but that's not relevant to the issue).\r\n\r\n```    \r\n   let trainLine = [|\r\n      for i in 0.0..0.01..18. -> \r\n            i, cos i|] \r\n\r\n   let testLine = [|\r\n      for i in 0.0..0.01..24. -> \r\n            i, cos i|] \r\n\r\n   let train = trainLine |> Array.map (fun (x,y) -> vector [D x] , vector [D y])\r\n \r\n   let net2 = createNetwork [|1;3; 1|]\r\n   let train2 = backprop true sigmoid net2 1e-3 0.005 10000 train \r\n\r\n   let errs = train2 |> Seq.takeOrMax 600 |> Seq.toArray\r\n```"},"comment":{"url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/comments/121269669","html_url":"https://github.com/DiffSharp/DiffSharp/issues/7#issuecomment-121269669","issue_url":"https://api.github.com/repos/DiffSharp/DiffSharp/issues/7","id":121269669,"user":{"login":"gbaydin","id":5365982,"avatar_url":"https://avatars.githubusercontent.com/u/5365982?v=3","gravatar_id":"","url":"https://api.github.com/users/gbaydin","html_url":"https://github.com/gbaydin","followers_url":"https://api.github.com/users/gbaydin/followers","following_url":"https://api.github.com/users/gbaydin/following{/other_user}","gists_url":"https://api.github.com/users/gbaydin/gists{/gist_id}","starred_url":"https://api.github.com/users/gbaydin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gbaydin/subscriptions","organizations_url":"https://api.github.com/users/gbaydin/orgs","repos_url":"https://api.github.com/users/gbaydin/repos","events_url":"https://api.github.com/users/gbaydin/events{/privacy}","received_events_url":"https://api.github.com/users/gbaydin/received_events","type":"User","site_admin":false},"created_at":"2015-07-14T14:55:09Z","updated_at":"2015-07-14T14:55:09Z","body":"Hi jragonslayer, thank you very much for reporting this!\r\n\r\nThere are three main things I should tell you:\r\n\r\n* For replicating the problem on my system and fixing it, could you please provide a full self-contained code, including your version of backprop and any other functions?\r\n\r\n* Are you running this as an fsx script or as a compiled executable? I've had stack overflow problems with F# in the past. Running code in F# Interactive, compiling with Debug or Build, or setting some compiler flags have effect on recursion limits. (Some relevant discussion: http://stackoverflow.com/questions/7947446/why-does-f-impose-a-low-limit-on-stack-size)\r\n\r\n* Most importantly, I should point out that the neural network code and the other examples in the \"Examples\" section are provided only as little demonstrations/tutorials on how to go about adding AD to your projects. They are VERY basic bare-bones implementations to show the basics of each method, kept simple and unoptimized for illustrative purposes. (I am actually working on a machine learning library built on top of DiffSharp, where neural networks and other things are fully implemented/tested and ready to work out of the box. I hope to release it soon.)\r\n"}}