{"action":"created","issue":{"url":"https://api.github.com/repos/soffes/ssziparchive/issues/75","labels_url":"https://api.github.com/repos/soffes/ssziparchive/issues/75/labels{/name}","comments_url":"https://api.github.com/repos/soffes/ssziparchive/issues/75/comments","events_url":"https://api.github.com/repos/soffes/ssziparchive/issues/75/events","html_url":"https://github.com/soffes/ssziparchive/issues/75","id":18874122,"number":75,"title":"problems for very large archives","user":{"login":"dbquarrel","id":5317653,"avatar_url":"https://avatars.githubusercontent.com/u/5317653?v=3","gravatar_id":"","url":"https://api.github.com/users/dbquarrel","html_url":"https://github.com/dbquarrel","followers_url":"https://api.github.com/users/dbquarrel/followers","following_url":"https://api.github.com/users/dbquarrel/following{/other_user}","gists_url":"https://api.github.com/users/dbquarrel/gists{/gist_id}","starred_url":"https://api.github.com/users/dbquarrel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dbquarrel/subscriptions","organizations_url":"https://api.github.com/users/dbquarrel/orgs","repos_url":"https://api.github.com/users/dbquarrel/repos","events_url":"https://api.github.com/users/dbquarrel/events{/privacy}","received_events_url":"https://api.github.com/users/dbquarrel/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/soffes/ssziparchive/labels/Bug","name":"Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2013-09-02T17:20:01Z","updated_at":"2015-04-06T20:27:08Z","closed_at":null,"body":"I need to unpack an archive which is very large in size, both number of files and the count of files. Overall the zip is 1.8 gigs with 26,000 files in it. \r\n\r\nSome suggestions:\r\n\r\n1. you need to use an @autoreleasepool {} structure in your main loop while you're yanking the individual files out of the zip. Otherwise all of the autoreleased objects that you're using will collect up on the main autorelease pool and the memory use will spiral out of control. Using this structure do { @autoreleasepool { } } while(i++ < 100000) has a much smaller memory footprint since it will nuke all of the autoreleased objects on each iteration. So I suggest you put those in.\r\n\r\n2. The mutable set directoriesModificationDates collects up all kinds of objects and then performs all of its actions at the end of the execution of the main loop. Again, in a mobile environment if you do this and someone encounters an archive like this (rare, but still), the set will just explode with objects causing the memory footprint to spiral way up. It is better instead to set the attributes on the fly as the objects are created, rather than collect them up into a set for processing later. This will keep your memory footprint with a known upper bound. \r\n\r\nBoth of these together implemented will mean you can process a zip archive of infinite size without ever causing the OS to nuke your process for excessive memory use. As is there are limits and SSZipArchive will get you nuked. Now, probably almost nobody will hit those limits... but I did. Might as well code it with no limits.\r\n\r\nThe side benefit is it's faster too than tracking everything in a datastructure for changing later.\r\n\r\nFurther suggestions:\r\n\r\nSimilarly this code constantly calls [self class] to get the class object to call a class method. Placing this call once at the top:\r\n\r\nclass myClass = [self class]\r\n\r\nWill in my case save something like 100,000 method calls. Tiny mods do a lot in a mobile environment.\r\n\r\nOh and also for delegates, rather than check every time you send a method whether or not they implement it, its better to check once, set a flag, and use that flag to make the call.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/soffes/ssziparchive/issues/comments/90230130","html_url":"https://github.com/soffes/ssziparchive/issues/75#issuecomment-90230130","issue_url":"https://api.github.com/repos/soffes/ssziparchive/issues/75","id":90230130,"user":{"login":"istx25","id":6799989,"avatar_url":"https://avatars.githubusercontent.com/u/6799989?v=3","gravatar_id":"","url":"https://api.github.com/users/istx25","html_url":"https://github.com/istx25","followers_url":"https://api.github.com/users/istx25/followers","following_url":"https://api.github.com/users/istx25/following{/other_user}","gists_url":"https://api.github.com/users/istx25/gists{/gist_id}","starred_url":"https://api.github.com/users/istx25/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/istx25/subscriptions","organizations_url":"https://api.github.com/users/istx25/orgs","repos_url":"https://api.github.com/users/istx25/repos","events_url":"https://api.github.com/users/istx25/events{/privacy}","received_events_url":"https://api.github.com/users/istx25/received_events","type":"User","site_admin":false},"created_at":"2015-04-06T20:27:08Z","updated_at":"2015-04-06T20:27:08Z","body":"Was this ever solved? (@dbquarrel @tnys @soffes)"}}