{"action":"created","issue":{"url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/50","labels_url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/50/labels{/name}","comments_url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/50/comments","events_url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/50/events","html_url":"https://github.com/lapwinglabs/x-ray/issues/50","id":85685317,"number":50,"title":"Paginate appears to cause multiple calls to composed function","user":{"login":"MaarekElets","id":6320290,"avatar_url":"https://avatars.githubusercontent.com/u/6320290?v=3","gravatar_id":"","url":"https://api.github.com/users/MaarekElets","html_url":"https://github.com/MaarekElets","followers_url":"https://api.github.com/users/MaarekElets/followers","following_url":"https://api.github.com/users/MaarekElets/following{/other_user}","gists_url":"https://api.github.com/users/MaarekElets/gists{/gist_id}","starred_url":"https://api.github.com/users/MaarekElets/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MaarekElets/subscriptions","organizations_url":"https://api.github.com/users/MaarekElets/orgs","repos_url":"https://api.github.com/users/MaarekElets/repos","events_url":"https://api.github.com/users/MaarekElets/events{/privacy}","received_events_url":"https://api.github.com/users/MaarekElets/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-06T03:04:24Z","updated_at":"2015-06-06T23:18:21Z","closed_at":null,"body":"Here's a partial example of the code I'm using (it's a fairly standard shopping page we are scraping for their benefit)\r\n\r\n```\r\nvar x = new xray();\r\n\t\tx(url, 'ul.product-grid', ['a@href'] )\r\n\t\t\t.paginate('.next-page a@href')\r\n\t\t\t(function(err, array) {\r\n\t\t\t\tconsole.log('--- BEGIN ARRAY LIST ---');\r\n\t\t\t\tconsole.log(array);\r\n\t\t\t\tconsole.log('--- END ARRAY LIST ---');\r\n                                ....launch other scraper that we use for page manipulation(I'd love to use the phantom-xray driver but I've not had any luck with it and the lack of .select (for select dropdowns) is a dealbreaker right now\r\n```\r\n\r\nWhat I end up seeing is my log message of begin, the first list of array hrefs, and end...then it processes the first couple of items (which happen to be skipped) and when it hits the first item using the other scraper I see the Begin Array List repeated but empty (there are items on the 2nd page). \r\n\r\nI'm not using the x-ray-phantom driver so I presume it's using the standard HTML driver, the other scraper uses Phantom. I can work with what I'm sure are bulk issues with the other package but I'm trying to determine both how paginate calls the composed function at the end (does it make 1 call per page?) and if this is the case, what is the proper way to call a function once the entire pagination process is complete (and my array of items is done)? I had the version 1 approach down pretty well, and given the newness of version 2 it's hard to separate \"working as intended\" from \"possibly a bug\". Any information you have would be very welcome. \r\n\r\n(BTW side note: if you ever were able to put together a page interaction example involving the x-ray-phantom driver and clicking/selecting [assuming it's even a use you intend] it would be a great thing. Appreciate your work on this. Thank you)\r\n\r\nEDIT: Caught a bad selector in paginate, however it didn't make a difference.\r\n"},"comment":{"url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/comments/109657205","html_url":"https://github.com/lapwinglabs/x-ray/issues/50#issuecomment-109657205","issue_url":"https://api.github.com/repos/lapwinglabs/x-ray/issues/50","id":109657205,"user":{"login":"matthewmueller","id":170299,"avatar_url":"https://avatars.githubusercontent.com/u/170299?v=3","gravatar_id":"","url":"https://api.github.com/users/matthewmueller","html_url":"https://github.com/matthewmueller","followers_url":"https://api.github.com/users/matthewmueller/followers","following_url":"https://api.github.com/users/matthewmueller/following{/other_user}","gists_url":"https://api.github.com/users/matthewmueller/gists{/gist_id}","starred_url":"https://api.github.com/users/matthewmueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthewmueller/subscriptions","organizations_url":"https://api.github.com/users/matthewmueller/orgs","repos_url":"https://api.github.com/users/matthewmueller/repos","events_url":"https://api.github.com/users/matthewmueller/events{/privacy}","received_events_url":"https://api.github.com/users/matthewmueller/received_events","type":"User","site_admin":false},"created_at":"2015-06-06T23:18:21Z","updated_at":"2015-06-06T23:18:21Z","body":"Oh interesting, can you provide the URL or HTML so I can debug it?\r\n\r\n"}}