{"action":"created","issue":{"url":"https://api.github.com/repos/jbenet/go-ipfs/issues/158","labels_url":"https://api.github.com/repos/jbenet/go-ipfs/issues/158/labels{/name}","comments_url":"https://api.github.com/repos/jbenet/go-ipfs/issues/158/comments","events_url":"https://api.github.com/repos/jbenet/go-ipfs/issues/158/events","html_url":"https://github.com/jbenet/go-ipfs/issues/158","id":45624308,"number":158,"title":"Add: recursion fix","user":{"login":"jbenet","id":138401,"avatar_url":"https://avatars.githubusercontent.com/u/138401?v=3","gravatar_id":"","url":"https://api.github.com/users/jbenet","html_url":"https://github.com/jbenet","followers_url":"https://api.github.com/users/jbenet/followers","following_url":"https://api.github.com/users/jbenet/following{/other_user}","gists_url":"https://api.github.com/users/jbenet/gists{/gist_id}","starred_url":"https://api.github.com/users/jbenet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbenet/subscriptions","organizations_url":"https://api.github.com/users/jbenet/orgs","repos_url":"https://api.github.com/users/jbenet/repos","events_url":"https://api.github.com/users/jbenet/events{/privacy}","received_events_url":"https://api.github.com/users/jbenet/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/jbenet/go-ipfs/labels/help+wanted","name":"help wanted","color":"159818"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2014-10-13T09:35:03Z","updated_at":"2015-01-17T09:11:42Z","closed_at":null,"body":"Currently subblocks are not added with addNode directly, but as part of their file dags. Currently, dir-file recursion happens via calls to `addNode`, and subblock recursion happens when `addNode` (on a file) calls `DAG.AddRecursive(nd)`. \r\n\r\nThe fact that `addNode` calls `DAG.AddRecursive(nd)` means it will try to add nodes again and again as `addNode` is called with parent dirs. The two alternatives are to either do everything with recursive backtracking, or generate the whole dag first, and _then_ call `DAG.AddRecursively(root)`. The former approach is less memory intensive (as the whole DAG does not need to be in memory) and parallelizes better, as can work on siblings simultaneously (without relying on the parallelism being inside `DAG.AddRecursively`).\r\n\r\nTwo related changes needed: \r\n\r\n1) The file-sublock recursion should be the same as dir-file recursion (i.e. calling `addNode` on the subblock).\r\n2) `addNode` should call `DAG.Add(nd)`\r\n"},"comment":{"url":"https://api.github.com/repos/jbenet/go-ipfs/issues/comments/70360170","html_url":"https://github.com/jbenet/go-ipfs/issues/158#issuecomment-70360170","issue_url":"https://api.github.com/repos/jbenet/go-ipfs/issues/158","id":70360170,"user":{"login":"whyrusleeping","id":1243164,"avatar_url":"https://avatars.githubusercontent.com/u/1243164?v=3","gravatar_id":"","url":"https://api.github.com/users/whyrusleeping","html_url":"https://github.com/whyrusleeping","followers_url":"https://api.github.com/users/whyrusleeping/followers","following_url":"https://api.github.com/users/whyrusleeping/following{/other_user}","gists_url":"https://api.github.com/users/whyrusleeping/gists{/gist_id}","starred_url":"https://api.github.com/users/whyrusleeping/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/whyrusleeping/subscriptions","organizations_url":"https://api.github.com/users/whyrusleeping/orgs","repos_url":"https://api.github.com/users/whyrusleeping/repos","events_url":"https://api.github.com/users/whyrusleeping/events{/privacy}","received_events_url":"https://api.github.com/users/whyrusleeping/received_events","type":"User","site_admin":false},"created_at":"2015-01-17T09:11:42Z","updated_at":"2015-01-17T09:11:42Z","body":"I dont think so... \r\n\r\nright here: https://github.com/jbenet/go-ipfs/blob/master/core/commands/add.go#L129\r\n\r\nwe recursively add the given node, when all of its children should have already been added to the DAG during `importer.BuildDagFromReader`. The simple fix to this *i think* is changing that call to not be recursive."}}