{"action":"created","issue":{"url":"https://api.github.com/repos/rgrove/sanitize/issues/129","labels_url":"https://api.github.com/repos/rgrove/sanitize/issues/129/labels{/name}","comments_url":"https://api.github.com/repos/rgrove/sanitize/issues/129/comments","events_url":"https://api.github.com/repos/rgrove/sanitize/issues/129/events","html_url":"https://github.com/rgrove/sanitize/issues/129","id":58469986,"number":129,"title":"Transformers memory leak?","user":{"login":"fabn","id":324213,"avatar_url":"https://avatars.githubusercontent.com/u/324213?v=3","gravatar_id":"","url":"https://api.github.com/users/fabn","html_url":"https://github.com/fabn","followers_url":"https://api.github.com/users/fabn/followers","following_url":"https://api.github.com/users/fabn/following{/other_user}","gists_url":"https://api.github.com/users/fabn/gists{/gist_id}","starred_url":"https://api.github.com/users/fabn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabn/subscriptions","organizations_url":"https://api.github.com/users/fabn/orgs","repos_url":"https://api.github.com/users/fabn/repos","events_url":"https://api.github.com/users/fabn/events{/privacy}","received_events_url":"https://api.github.com/users/fabn/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rgrove/sanitize/labels/bug","name":"bug","color":"ff0000"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-02-21T18:45:40Z","updated_at":"2015-02-22T12:00:08Z","closed_at":null,"body":"I'm using your gem to fix a very badly formatted markup. I'm running the cleaning process in a sidekiq worker and after a while process memory grows with no limit. I (think I have) tracked down the leak to `Sanitize.fragment` call. I made a simple script which should reproduce the issue.\r\n\r\n```ruby\r\n200_000.times do |i|\r\n  Sanitize.fragment(raw_markup, Sanitize::Config::WIKIPEDIA)\r\n  if i % 10 == 0\r\n    GC.start\r\n    ending = GC.stat\r\n    puts \"#{i} Referenced objects count is: #{ending[:total_allocated_object] - ending[:total_freed_object]}\"\r\n    puts \"#{i} Current used memory is #{`ps -o rss= -p #{Process.pid}`.to_i / 1024}MB\"\r\n    puts ObjectSpace.count_objects\r\n    puts \"\\n\"\r\n  end\r\nend\r\n```\r\n\r\nIf i run this code on my local machine at iteration 3260 used memory is 813MB and it keeps growing, while other dumped params don't change across iterations. If I use one of the predefined configurations it doesn't happen <sup><a href=\"#note\">1</a></sup>.\r\n\r\nHere is my sanitize config \r\n\r\n```ruby\r\nclass Sanitize\r\n  module Config\r\n    # Very restrictive config for wikipedia markup sanitizer\r\n    WIKIPEDIA = freeze_config(\r\n        elements: %w[b em i strong u br ol ul li pre p h1 h2 h3 h4 h5 h6],\r\n        remove_contents: %w[sup table],\r\n        # Transformers section\r\n        transformers: [\r\n            # Remove wikipedia edit links\r\n            lambda { |env|\r\n              # Completely remove span.editsection nodes\r\n              if env[:node_name] == 'span' && /editsection/ =~ env[:node][:class]\r\n                env[:node].unlink\r\n              end\r\n            },\r\n            # Remove wikipedia thumb divs i.e. div with some graphic and caption\r\n            lambda { |env|\r\n              if env[:node_name] == 'div' && /thumb/ =~ env[:node][:class]\r\n                env[:node].unlink\r\n              end\r\n            },\r\n        ]\r\n    )\r\n  end\r\nend\r\n```\r\n\r\nHere are my current bundle (relevant gems)\r\n\r\n```\r\nGems included by the bundle:\r\n  * crass (1.0.1)\r\n  * nokogiri (1.6.6.2)\r\n  * nokogumbo (1.2.0)\r\n  * sanitize (3.1.1)\r\n```\r\n\r\nFull code including the html sample used is available here: https://gist.github.com/fabn/304691f9bd81e1b814cf\r\n\r\n<a name=\"note\">(1)</a>: Actually memory usage grows also with `RELAXED` config but very very slowly (i.e. after 4000 iterations used memory is ~70MB while value reported at first iteration was 25MB)"},"comment":{"url":"https://api.github.com/repos/rgrove/sanitize/issues/comments/75432263","html_url":"https://github.com/rgrove/sanitize/issues/129#issuecomment-75432263","issue_url":"https://api.github.com/repos/rgrove/sanitize/issues/129","id":75432263,"user":{"login":"fabn","id":324213,"avatar_url":"https://avatars.githubusercontent.com/u/324213?v=3","gravatar_id":"","url":"https://api.github.com/users/fabn","html_url":"https://github.com/fabn","followers_url":"https://api.github.com/users/fabn/followers","following_url":"https://api.github.com/users/fabn/following{/other_user}","gists_url":"https://api.github.com/users/fabn/gists{/gist_id}","starred_url":"https://api.github.com/users/fabn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabn/subscriptions","organizations_url":"https://api.github.com/users/fabn/orgs","repos_url":"https://api.github.com/users/fabn/repos","events_url":"https://api.github.com/users/fabn/events{/privacy}","received_events_url":"https://api.github.com/users/fabn/received_events","type":"User","site_admin":false},"created_at":"2015-02-22T12:00:08Z","updated_at":"2015-02-22T12:00:08Z","body":"Thanks for your speedy support.\r\n\r\nI tried the same code with dev-3.1.2 branch and it's much better. To give you some numbers at the same iteration I reported earlier memory usage went down from 813MB (in 3.1.1) to 167MB with 3.1.2. So your change makes a great improvement.\r\n\r\nHowever I still see memory growing during time, so there should be some leak in nokogiri itself. I found for instance this report: https://github.com/sparklemotion/nokogiri/issues/1063"}}