{"action":"created","issue":{"url":"https://api.github.com/repos/meh/ruby-mbox/issues/22","labels_url":"https://api.github.com/repos/meh/ruby-mbox/issues/22/labels{/name}","comments_url":"https://api.github.com/repos/meh/ruby-mbox/issues/22/comments","events_url":"https://api.github.com/repos/meh/ruby-mbox/issues/22/events","html_url":"https://github.com/meh/ruby-mbox/issues/22","id":101282628,"number":22,"title":"performance not so great on large files","user":{"login":"coldnebo","id":73004,"avatar_url":"https://avatars.githubusercontent.com/u/73004?v=3","gravatar_id":"","url":"https://api.github.com/users/coldnebo","html_url":"https://github.com/coldnebo","followers_url":"https://api.github.com/users/coldnebo/followers","following_url":"https://api.github.com/users/coldnebo/following{/other_user}","gists_url":"https://api.github.com/users/coldnebo/gists{/gist_id}","starred_url":"https://api.github.com/users/coldnebo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/coldnebo/subscriptions","organizations_url":"https://api.github.com/users/coldnebo/orgs","repos_url":"https://api.github.com/users/coldnebo/repos","events_url":"https://api.github.com/users/coldnebo/events{/privacy}","received_events_url":"https://api.github.com/users/coldnebo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-08-16T17:33:01Z","updated_at":"2015-08-16T17:58:44Z","closed_at":null,"body":"parsing a 32M mbox file promptly hung on this line (from ruby trace):\r\n\r\n```\r\n#0:/Library/Ruby/Gems/2.0.0/gems/mbox-0.1.0/lib/mbox/mail/content.rb:38:Mbox::Mail::Content:-: \t\tif type && type.mime && type.boundary && matches = type.mime.match(%r{multipart/(\\w+)})\r\n#0:/Library/Ruby/Gems/2.0.0/gems/mbox-0.1.0/lib/mbox/mail/content.rb:39:Mbox::Mail::Content:-: \t\t\ttext.sub(/^.*?--#{type.boundary}\\n/m, '').sub(/--#{type.boundary}--$/m, '').split(\"--#{type.boundary}\\n\").each {|part|\r\n^C#0:/Library/Ruby/Gems/2.0.0/gems/mbox-0.1.0/lib/mbox/mail/content.rb:39:String:^: \t\t\ttext.sub(/^.*?--#{type.boundary}\\n/m, '').sub(/--#{type.boundary}--$/m, '').split(\"--#{type.boundary}\\n\").each {|part|\r\n#0:/Library/Ruby/Gems/2.0.0/gems/mbox-0.1.0/lib/mbox/mail/content.rb:39:Mbox::Mail::Content:<: \t\t\ttext.sub(/^.*?--#{type.boundary}\\n/m, '').sub(/--#{type.boundary}--$/m, '').split(\"--#{type.boundary}\\n\").each {|part|\r\n#\r\n```\r\n\r\nIt's not clear from the doc whether there is a way of paging over messages with large attachments.  I'm using the stream interface, but it appears to want to parse the entire email message on most operations.\r\n\r\nRunning through tracer, It seems that split line is creating a large number of substrings... maybe an attached pdf or something.\r\n\r\nGoing to take a look at node-mbox in the meantime for a faster streaming solution."},"comment":{"url":"https://api.github.com/repos/meh/ruby-mbox/issues/comments/131595213","html_url":"https://github.com/meh/ruby-mbox/issues/22#issuecomment-131595213","issue_url":"https://api.github.com/repos/meh/ruby-mbox/issues/22","id":131595213,"user":{"login":"meh","id":40204,"avatar_url":"https://avatars.githubusercontent.com/u/40204?v=3","gravatar_id":"","url":"https://api.github.com/users/meh","html_url":"https://github.com/meh","followers_url":"https://api.github.com/users/meh/followers","following_url":"https://api.github.com/users/meh/following{/other_user}","gists_url":"https://api.github.com/users/meh/gists{/gist_id}","starred_url":"https://api.github.com/users/meh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meh/subscriptions","organizations_url":"https://api.github.com/users/meh/orgs","repos_url":"https://api.github.com/users/meh/repos","events_url":"https://api.github.com/users/meh/events{/privacy}","received_events_url":"https://api.github.com/users/meh/received_events","type":"User","site_admin":false},"created_at":"2015-08-16T17:58:44Z","updated_at":"2015-08-16T17:58:44Z","body":"You can pass a parameter to `#each` to request it only reads headers and not contents: ```mbox.each(headers_only: true)```."}}