{"action":"created","issue":{"url":"https://api.github.com/repos/libextract/libextract/issues/1","labels_url":"https://api.github.com/repos/libextract/libextract/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/libextract/libextract/issues/1/comments","events_url":"https://api.github.com/repos/libextract/libextract/issues/1/events","html_url":"https://github.com/libextract/libextract/issues/1","id":66593424,"number":1,"title":"Pipeline or Strategy style?","user":{"login":"eugene-eeo","id":3669236,"avatar_url":"https://avatars.githubusercontent.com/u/3669236?v=3","gravatar_id":"","url":"https://api.github.com/users/eugene-eeo","html_url":"https://github.com/eugene-eeo","followers_url":"https://api.github.com/users/eugene-eeo/followers","following_url":"https://api.github.com/users/eugene-eeo/following{/other_user}","gists_url":"https://api.github.com/users/eugene-eeo/gists{/gist_id}","starred_url":"https://api.github.com/users/eugene-eeo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eugene-eeo/subscriptions","organizations_url":"https://api.github.com/users/eugene-eeo/orgs","repos_url":"https://api.github.com/users/eugene-eeo/repos","events_url":"https://api.github.com/users/eugene-eeo/events{/privacy}","received_events_url":"https://api.github.com/users/eugene-eeo/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/libextract/libextract/labels/help%20wanted","name":"help wanted","color":"159818"}],"state":"open","locked":false,"assignee":{"login":"eugene-eeo","id":3669236,"avatar_url":"https://avatars.githubusercontent.com/u/3669236?v=3","gravatar_id":"","url":"https://api.github.com/users/eugene-eeo","html_url":"https://github.com/eugene-eeo","followers_url":"https://api.github.com/users/eugene-eeo/followers","following_url":"https://api.github.com/users/eugene-eeo/following{/other_user}","gists_url":"https://api.github.com/users/eugene-eeo/gists{/gist_id}","starred_url":"https://api.github.com/users/eugene-eeo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eugene-eeo/subscriptions","organizations_url":"https://api.github.com/users/eugene-eeo/orgs","repos_url":"https://api.github.com/users/eugene-eeo/repos","events_url":"https://api.github.com/users/eugene-eeo/events{/privacy}","received_events_url":"https://api.github.com/users/eugene-eeo/received_events","type":"User","site_admin":false},"milestone":null,"comments":11,"created_at":"2015-04-06T12:21:14Z","updated_at":"2015-04-09T16:08:27Z","closed_at":null,"body":"Currently there are two methods of extraction that I have implemented, pipeline (function based) and strategy (inheritance based) style. I am not quite sure which style to use, any suggestions?\r\n\r\n### Pipeline style\r\n\r\n```python\r\ndef custom_get_text(node):\r\n    return node.xpath(SELECT_TEXT)\r\n\r\nstrategy = list(html.STRATEGY)\r\nstrategy[-1] = custom_get_text\r\n```\r\n\r\n**Pros:**\r\n - Higher performance (in theory) as functions are easier to optimise\r\n - No shared state, easy to debug compared to OO approach\r\n - More elegant looking, more expressive (strategies can just be ``[get_nodepairs, get_best, get_text]``)\r\n - Code as data (pipelines are just lists)\r\n\r\n**Cons:**\r\n - May be slower due to use of many functions\r\n - Not configurable (at the moment)\r\n\r\n### Strategy style\r\n\r\n```python\r\nclass CustomFinder(HTMLFinder):\r\n    text_nodes = 'custom_selector'\r\n```\r\n\r\n**Pros:**\r\n - More configurable and easier to configure\r\n - Most developers are more familiar\r\n - Shared state can be used for optimisation\r\n\r\n**Cons:**\r\n - Hard to test and debug\r\n - May be slower due to opaque objects\r\n\r\n**cc @rodricios**"},"comment":{"url":"https://api.github.com/repos/libextract/libextract/issues/comments/91276204","html_url":"https://github.com/libextract/libextract/issues/1#issuecomment-91276204","issue_url":"https://api.github.com/repos/libextract/libextract/issues/1","id":91276204,"user":{"login":"Beluki","id":6313150,"avatar_url":"https://avatars.githubusercontent.com/u/6313150?v=3","gravatar_id":"","url":"https://api.github.com/users/Beluki","html_url":"https://github.com/Beluki","followers_url":"https://api.github.com/users/Beluki/followers","following_url":"https://api.github.com/users/Beluki/following{/other_user}","gists_url":"https://api.github.com/users/Beluki/gists{/gist_id}","starred_url":"https://api.github.com/users/Beluki/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Beluki/subscriptions","organizations_url":"https://api.github.com/users/Beluki/orgs","repos_url":"https://api.github.com/users/Beluki/repos","events_url":"https://api.github.com/users/Beluki/events{/privacy}","received_events_url":"https://api.github.com/users/Beluki/received_events","type":"User","site_admin":false},"created_at":"2015-04-09T16:08:27Z","updated_at":"2015-04-09T16:08:27Z","body":"Hi there! May I chime in to give an opinion?\r\n\r\nI prefer the object-oriented approach in this particular case.\r\n\r\nMy rationale is that it makes it easier to add things such as pretty-printing or tracing steps in the middle of the computation. This helps debugging.\r\n\r\nAnother option that is somewhere in between both approaches and allows configuration is using closures. Made-up example as I don't know anything about libextract's internals:\r\n\r\n```python\r\ndef score_higher_than(value):\r\n    def custom(node):\r\n         return node.score > value\r\n    return custom\r\n\r\nSTRATEGY = ( score_higher_than(20), get_etree, get_pairs )\r\n```\r\n\r\nOn performance:\r\n\r\nWhen I wrote [MQLite](https://github.com/Beluki/MQLite), I heavily benchmarked all of those approaches: single functions, classes and closures.\r\n\r\nIn CPython I found functions and closures to be faster than classes (sometimes 2x faster depending on how many applications are needed). Using `__call__` to emulate functions was also slower than having a dedicated method in the class. In Pypy the difference was non-existant for all practical purposes.\r\n\r\nI also found that as soon as the classes/functions do anything non-trivial, the performance difference becomes negligible. In my program, I create trees of matching classes and call methods from them thousands of times (as many as there are input records). The performance hit is basically 0 when compared to the work the methods actually do."}}