{"action":"created","issue":{"url":"https://api.github.com/repos/cantino/huginn/issues/873","labels_url":"https://api.github.com/repos/cantino/huginn/issues/873/labels{/name}","comments_url":"https://api.github.com/repos/cantino/huginn/issues/873/comments","events_url":"https://api.github.com/repos/cantino/huginn/issues/873/events","html_url":"https://github.com/cantino/huginn/issues/873","id":90345374,"number":873,"title":"Adding \"next\" page navigation to WebsiteAgent","user":{"login":"knu","id":10236,"avatar_url":"https://avatars.githubusercontent.com/u/10236?v=3","gravatar_id":"","url":"https://api.github.com/users/knu","html_url":"https://github.com/knu","followers_url":"https://api.github.com/users/knu/followers","following_url":"https://api.github.com/users/knu/following{/other_user}","gists_url":"https://api.github.com/users/knu/gists{/gist_id}","starred_url":"https://api.github.com/users/knu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knu/subscriptions","organizations_url":"https://api.github.com/users/knu/orgs","repos_url":"https://api.github.com/users/knu/repos","events_url":"https://api.github.com/users/knu/events{/privacy}","received_events_url":"https://api.github.com/users/knu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-23T10:14:19Z","updated_at":"2015-06-23T14:51:03Z","closed_at":null,"body":"I've been thinking of adding a key named \"next\" to WebsiteAgent so you can tell an agent to navigate to the next page(s) if existent.\r\n\r\nFor instance, you could specify the following:\r\n```\r\n\"next\": {\r\n  \"css\": \"a[rel~='next']\",\r\n  \"value\": \"@href\"\r\n}\r\n```\r\nAnd if the fetched page had a link with a `rel` attribute value including \"next\", then the agent would navigate to the link URL.  This would be repeated until there was no \"next\" link. \r\n\r\nThere should also be a way to stop an iteration, like when it reaches the point where it has already seen, or when it is to emit an event that is identical to any existing (recent) event.  I'm yet to decide how to let user specify the conditions.  Should it provide a way to save and access `memory` for ultimate programmability or is that an overkill?\r\n\r\nI'll soon be working on it, so please share your ideas if you are interested."},"comment":{"url":"https://api.github.com/repos/cantino/huginn/issues/comments/114532658","html_url":"https://github.com/cantino/huginn/issues/873#issuecomment-114532658","issue_url":"https://api.github.com/repos/cantino/huginn/issues/873","id":114532658,"user":{"login":"cantino","id":83835,"avatar_url":"https://avatars.githubusercontent.com/u/83835?v=3","gravatar_id":"","url":"https://api.github.com/users/cantino","html_url":"https://github.com/cantino","followers_url":"https://api.github.com/users/cantino/followers","following_url":"https://api.github.com/users/cantino/following{/other_user}","gists_url":"https://api.github.com/users/cantino/gists{/gist_id}","starred_url":"https://api.github.com/users/cantino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cantino/subscriptions","organizations_url":"https://api.github.com/users/cantino/orgs","repos_url":"https://api.github.com/users/cantino/repos","events_url":"https://api.github.com/users/cantino/events{/privacy}","received_events_url":"https://api.github.com/users/cantino/received_events","type":"User","site_admin":false},"created_at":"2015-06-23T14:51:03Z","updated_at":"2015-06-23T14:51:03Z","body":"Cool!  Would just a selector and a max paginations be enough, when combined\nwith the normal event deduplication?\n\nOn Tuesday, June 23, 2015, Akinori MUSHA <notifications@github.com> wrote:\n\n> I've been thinking of adding a key named \"next\" to WebsiteAgent so you can\n> tell an agent to navigate to the next page(s) if existent.\n>\n> For instance, you could specify the following:\n>\n> \"next\": {\n>   \"css\": \"a[rel~='next']\",\n>   \"value\": \"@href\"\n> }\n>\n> And if the fetched page had a link with a rel attribute value including\n> \"next\", then the agent would navigate to the link URL. This would be\n> repeated until there was no \"next\" link.\n>\n> There should also be a way to stop an iteration, like when it reaches the\n> point where it has already seen, or when it is to emit an event that is\n> identical to an existing (recent) event. I'm yet to decide how to let user\n> specify the conditions. Should it provide a way to save and access memory\n> for ultimate programmability or is that an overkill?\n>\n> I'll soon be working on it, so please share your ideas if you are\n> interested.\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/cantino/huginn/issues/873>.\n>\n"}}