{"action":"created","issue":{"url":"https://api.github.com/repos/simonmichael/hledger/issues/275","labels_url":"https://api.github.com/repos/simonmichael/hledger/issues/275/labels{/name}","comments_url":"https://api.github.com/repos/simonmichael/hledger/issues/275/comments","events_url":"https://api.github.com/repos/simonmichael/hledger/issues/275/events","html_url":"https://github.com/simonmichael/hledger/issues/275","id":96372338,"number":275,"title":"hledger 0.26 runs out of memory with large(?) ledger file(s)","user":{"login":"MarkusTeufelberger","id":950561,"avatar_url":"https://avatars.githubusercontent.com/u/950561?v=3","gravatar_id":"","url":"https://api.github.com/users/MarkusTeufelberger","html_url":"https://github.com/MarkusTeufelberger","followers_url":"https://api.github.com/users/MarkusTeufelberger/followers","following_url":"https://api.github.com/users/MarkusTeufelberger/following{/other_user}","gists_url":"https://api.github.com/users/MarkusTeufelberger/gists{/gist_id}","starred_url":"https://api.github.com/users/MarkusTeufelberger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarkusTeufelberger/subscriptions","organizations_url":"https://api.github.com/users/MarkusTeufelberger/orgs","repos_url":"https://api.github.com/users/MarkusTeufelberger/repos","events_url":"https://api.github.com/users/MarkusTeufelberger/events{/privacy}","received_events_url":"https://api.github.com/users/MarkusTeufelberger/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/simonmichael/hledger/labels/balance","name":"balance","color":"bfd4f2"},{"url":"https://api.github.com/repos/simonmichael/hledger/labels/BUG","name":"BUG","color":"fc2929"},{"url":"https://api.github.com/repos/simonmichael/hledger/labels/performance","name":"performance","color":"bfd4f2"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-07-21T17:28:47Z","updated_at":"2015-07-22T16:19:08Z","closed_at":null,"body":"I have a ledger of about 40 MB worth of files, hledger 0.26 now suddenly chokes on them, the previous version worked fine (well it also took waaay to long to calculate balances, but at least it worked):\r\n\r\n    hledger bal -f ./output/full.ledger\r\n    hledger: out of memory\r\n\r\nfull.ledger just imports a couple of other files (I have a collection of scripts that generate them from CSV files) that total to 37 MB. My machine has 16 GB of RAM, so this is not very likely an issue.\r\n\r\n<bountysource-plugin>\r\n---\r\nWant to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/24719851-hledger-0-26-runs-out-of-memory-with-large-ledger-file-s?utm_campaign=plugin&utm_content=tracker%2F536505&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F536505&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>"},"comment":{"url":"https://api.github.com/repos/simonmichael/hledger/issues/comments/123777892","html_url":"https://github.com/simonmichael/hledger/issues/275#issuecomment-123777892","issue_url":"https://api.github.com/repos/simonmichael/hledger/issues/275","id":123777892,"user":{"login":"simonmichael","id":10144,"avatar_url":"https://avatars.githubusercontent.com/u/10144?v=3","gravatar_id":"","url":"https://api.github.com/users/simonmichael","html_url":"https://github.com/simonmichael","followers_url":"https://api.github.com/users/simonmichael/followers","following_url":"https://api.github.com/users/simonmichael/following{/other_user}","gists_url":"https://api.github.com/users/simonmichael/gists{/gist_id}","starred_url":"https://api.github.com/users/simonmichael/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simonmichael/subscriptions","organizations_url":"https://api.github.com/users/simonmichael/orgs","repos_url":"https://api.github.com/users/simonmichael/repos","events_url":"https://api.github.com/users/simonmichael/events{/privacy}","received_events_url":"https://api.github.com/users/simonmichael/received_events","type":"User","site_admin":false},"created_at":"2015-07-22T16:19:08Z","updated_at":"2015-07-22T16:19:08Z","body":"Hmm. I didn't see this in pre-release testing, and I don't see it now with the simple test journals. Time is the same:\r\n```\r\n$ time tools/simplebench -n1 hledger-0.24 hledger-0.25 hledger-0.26\r\nUsing bench.tests\r\nRunning 6 tests 1 times with 3 executables at 2015-07-22 13:53:31.42249 UTC:\r\n\r\nSummary (best iteration):\r\n\r\n+------------------------------------++--------------+--------------+--------------+\r\n|                                    || hledger-0.24 | hledger-0.25 | hledger-0.26 |\r\n+====================================++==============+==============+==============+\r\n| bal -f data/100x100x10.journal     ||         0.04 |         0.04 |         0.04 |\r\n| bal -f data/1000x1000x10.journal   ||         0.21 |         0.25 |         0.24 |\r\n| bal -f data/1000x10000x10.journal  ||         0.31 |         0.32 |         0.35 |\r\n| bal -f data/10000x1000x10.journal  ||         1.76 |         1.72 |         1.85 |\r\n| bal -f data/10000x10000x10.journal ||         3.18 |         3.38 |         3.58 |\r\n| bal -f data/100000x1000x10.journal ||        18.69 |        20.74 |        19.79 |\r\n+------------------------------------++--------------+--------------+--------------+\r\n\r\n\r\nreal\t1m16.674s\r\nuser\t1m10.822s\r\nsys\t0m3.867s\r\n```\r\nand more importantly, the heap profile looks the same:\r\n\r\n0.25.1:\r\n![hledgerprof 0 25 1](https://cloud.githubusercontent.com/assets/10144/8830698/7bf329a8-3052-11e5-9ba4-77242156f8c2.png)\r\n\r\n0.26:\r\n![hledgerprof 0 26](https://cloud.githubusercontent.com/assets/10144/8830699/7c36e59e-3052-11e5-8d9c-740191db1e89.png)\r\n\r\nWhat special features does your data have ? Maybe an unusually high number of aliases, accounts, includes, or something else ? Would it be possible to simplify it until you see the problem go away ?"}}