{"action":"opened","issue":{"url":"https://api.github.com/repos/wfp/node-html-auditor/issues/38","labels_url":"https://api.github.com/repos/wfp/node-html-auditor/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/wfp/node-html-auditor/issues/38/comments","events_url":"https://api.github.com/repos/wfp/node-html-auditor/issues/38/events","html_url":"https://github.com/wfp/node-html-auditor/issues/38","id":123431583,"number":38,"title":"timeout issue","user":{"login":"lukasa1993","id":1499012,"avatar_url":"https://avatars.githubusercontent.com/u/1499012?v=3","gravatar_id":"","url":"https://api.github.com/users/lukasa1993","html_url":"https://github.com/lukasa1993","followers_url":"https://api.github.com/users/lukasa1993/followers","following_url":"https://api.github.com/users/lukasa1993/following{/other_user}","gists_url":"https://api.github.com/users/lukasa1993/gists{/gist_id}","starred_url":"https://api.github.com/users/lukasa1993/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lukasa1993/subscriptions","organizations_url":"https://api.github.com/users/lukasa1993/orgs","repos_url":"https://api.github.com/users/lukasa1993/repos","events_url":"https://api.github.com/users/lukasa1993/events{/privacy}","received_events_url":"https://api.github.com/users/lukasa1993/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-12-22T07:32:07Z","updated_at":"2015-12-22T07:32:07Z","closed_at":null,"body":"Beside http timeout there is generally good idea to background this whole process but lets only talk about problem.\r\n\r\nso main problem is that when we check for link or make request to it, on other side webserver might not be responding for whatever reason, so at this point our http client (basically any http client) waits till it times out request. so why we care about this.\r\n\r\nour use case of this modules is to call all for 4 steps (will define them below) from cron job, or even from browser button click problem still holds\r\n\r\nto flash out 4 steps:\r\n- http-fetch we grab sitemaps (xml) \r\n\t- all other steps are dependent on this, and after this may run in parallel\r\n- a11y-audit\r\n- html5-audit\r\n- link-audit this waits for webserver to respond to check if its alive and 2 ways its dead is:\r\n\t- webserver didn't respond at all\r\n\t- webserver returned http code that link doesn't exist \r\n\r\nso as we said we call this from cron job or from click in webpage in both cases we end up with problem when we need to wait for 1 one to finish and then run all 3 and then get back with result or log error in watchdog, to achieve this naturally we need threads but big wall here is that PHP doesn't have threads whats so ever.\r\n\r\nso we need to come up with some workaround, next will list all the possible solutions we went through (and failed):  \r\n\r\n- we search for thread solution in Drupal and in symphony to have some lib for background processes\r\n  - all we found was either in D7 or not finished \r\n- then we tried to use cron to handle this but for that to work we need to set cron to run in every 1 minute which is not desirable so we abandoned this idea\r\n- last was D8 QueryWorker we hoped it was solving this in manner we described but in the end it appeared to be queueing in cron, so each queue would run on next cron run so basically to do all this 4 steps about week will be needed\r\n\r\nso here are solutions we still have, only problem with them is they are all costume (not Drupal way, no symphony way) for PHP but its also some logical because PHP just don't have threading:\r\n\r\n* we can pass our DB config parameter as JSON to NodeJS processes and in case of error log them in watchdog, and limit execution time on NodeJS side \r\n * we already log result in filesystem and we accepted that flow so only problem is error\r\n * problem here is that we might not be comfortable to run rouge NodeJS process and hope it will end itself even with timeout \r\n* second version is to come up with costume threading in php and implement original solution but this is really long path and might end up a bug hole if not done properly \r\n\r\n\r\nto conclude this is all i think we got, any limits or partly completing job on single PHP timeout is just hoping that all goes well and thats it it won't be a solid solution"}}