{"action":"created","issue":{"url":"https://api.github.com/repos/pyload/pyload/issues/1191","labels_url":"https://api.github.com/repos/pyload/pyload/issues/1191/labels{/name}","comments_url":"https://api.github.com/repos/pyload/pyload/issues/1191/comments","events_url":"https://api.github.com/repos/pyload/pyload/issues/1191/events","html_url":"https://github.com/pyload/pyload/issues/1191","id":58205851,"number":1191,"title":"ZippyShare keeps changing its checksum","user":{"login":"sebdelsol","id":6039281,"avatar_url":"https://avatars.githubusercontent.com/u/6039281?v=3","gravatar_id":"","url":"https://api.github.com/users/sebdelsol","html_url":"https://github.com/sebdelsol","followers_url":"https://api.github.com/users/sebdelsol/followers","following_url":"https://api.github.com/users/sebdelsol/following{/other_user}","gists_url":"https://api.github.com/users/sebdelsol/gists{/gist_id}","starred_url":"https://api.github.com/users/sebdelsol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sebdelsol/subscriptions","organizations_url":"https://api.github.com/users/sebdelsol/orgs","repos_url":"https://api.github.com/users/sebdelsol/repos","events_url":"https://api.github.com/users/sebdelsol/events{/privacy}","received_events_url":"https://api.github.com/users/sebdelsol/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pyload/pyload/labels/plugin%20issue","name":"plugin issue","color":"eb6420"},{"url":"https://api.github.com/repos/pyload/pyload/labels/testing","name":"testing","color":"006b75"}],"state":"open","locked":false,"assignee":{"login":"vuolter","id":1221092,"avatar_url":"https://avatars.githubusercontent.com/u/1221092?v=3","gravatar_id":"","url":"https://api.github.com/users/vuolter","html_url":"https://github.com/vuolter","followers_url":"https://api.github.com/users/vuolter/followers","following_url":"https://api.github.com/users/vuolter/following{/other_user}","gists_url":"https://api.github.com/users/vuolter/gists{/gist_id}","starred_url":"https://api.github.com/users/vuolter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vuolter/subscriptions","organizations_url":"https://api.github.com/users/vuolter/orgs","repos_url":"https://api.github.com/users/vuolter/repos","events_url":"https://api.github.com/users/vuolter/events{/privacy}","received_events_url":"https://api.github.com/users/vuolter/received_events","type":"User","site_admin":false},"milestone":null,"comments":7,"created_at":"2015-02-19T13:08:25Z","updated_at":"2015-02-22T15:15:13Z","closed_at":null,"body":"Based on what @gaberad suggested, I have Zippyshare working for at least the 4 last changes. \r\n\r\nBody's JS scripts are evaluated with all the DOM statements turned into regular variables initialized with their fetched values in the html.\r\n```python\r\n# get all the scripts inside the html body\r\ntree = lxml.html.fromstring(self.html)\r\nscripts = tree.xpath('//body//script[@type=\"text/javascript\"]/text()')\r\n\r\n# meant to be populated with the initialization of all the DOM elements found in the scripts\r\ninitScripts = []\r\n\r\ndef repl(element):\r\n    id, attr = element.group(1), element.group(4)      # attr might be None\r\n\r\n    varName = '%s_%s' %(id, attr)\r\n    initValue = tree.xpath('//*[@id=\"%s\"]/@%s' %(id, attr))\r\n    initValue = '\"%s\"' %initValue[-1] if initValue else 'null'\r\n\r\n    initScripts.append('var %s = %s;' %(varName, initValue))\r\n    return varName\r\n\r\n# handle all getElementById\r\nreVar = r'document.getElementById\\([\\'\"](\\w+)[\\'\"]\\)(\\.)?(getAttribute\\([\\'\"])?(\\w+)?([\\'\"]\\))?'\r\nscripts = [re.sub(reVar, repl, script) for script in scripts]\r\n\r\n# get the file's url by evaluating all the scripts\r\nreturn  JSE_chainEval(initScripts + scripts + ['dlbutton_href'])\r\n```\r\n\r\nSome JS scripts are meant to fail at some point, hence  `JSE_chainEval()` needs to catch errors and resume the evaluation of all the remaining scripts with *the same JS context*:\r\n\r\n```python\r\ndef JSE_chainEval(scripts):\r\n    rt = PyV8.JSContext()\r\n    rt.enter()\r\n\r\n    for script in scripts:\r\n        try:\r\n            ret = rt.eval(script.encode('utf8') if isinstance(script, unicode) else script)\r\n        except:\r\n            pass\r\n\r\n    return ret\r\n```\r\nEDIT : I don't know how to use xpath (or beautifulSoup) in a hoster. So I just assumed that lxml.html would be available in some way...\r\n\r\nEDIT2: ZippyShare changed again! I had to handle some DOM introspection, eg. document.getElementById('omg').getAttribute('class')"},"comment":{"url":"https://api.github.com/repos/pyload/pyload/issues/comments/75439804","html_url":"https://github.com/pyload/pyload/issues/1191#issuecomment-75439804","issue_url":"https://api.github.com/repos/pyload/pyload/issues/1191","id":75439804,"user":{"login":"stickell","id":335415,"avatar_url":"https://avatars.githubusercontent.com/u/335415?v=3","gravatar_id":"","url":"https://api.github.com/users/stickell","html_url":"https://github.com/stickell","followers_url":"https://api.github.com/users/stickell/followers","following_url":"https://api.github.com/users/stickell/following{/other_user}","gists_url":"https://api.github.com/users/stickell/gists{/gist_id}","starred_url":"https://api.github.com/users/stickell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stickell/subscriptions","organizations_url":"https://api.github.com/users/stickell/orgs","repos_url":"https://api.github.com/users/stickell/repos","events_url":"https://api.github.com/users/stickell/events{/privacy}","received_events_url":"https://api.github.com/users/stickell/received_events","type":"User","site_admin":false},"created_at":"2015-02-22T15:15:13Z","updated_at":"2015-02-22T15:15:13Z","body":"No we will not, we can't add anything directly in 0.4.9 we should wait next release but we are willing to abandon this approach. Now we prefer making users install their own using PIP as they already have to do for other pyLoad core deps as pyCurl for example.\r\n\r\nUse a `try` statement on the import to print a message on the console to inform users they have to install `lxml` if they want to use this plugin. This is not the first plugin that is using external deps and this approach should work quite well."}}