{"action":"created","issue":{"url":"https://api.github.com/repos/hypothesis/via/issues/9","labels_url":"https://api.github.com/repos/hypothesis/via/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/hypothesis/via/issues/9/comments","events_url":"https://api.github.com/repos/hypothesis/via/issues/9/events","html_url":"https://github.com/hypothesis/via/issues/9","id":71113705,"number":9,"title":"Loading Hypothesis doesn't work on pages without <head>","user":{"login":"janpaul123","id":177461,"avatar_url":"https://avatars.githubusercontent.com/u/177461?v=3","gravatar_id":"","url":"https://api.github.com/users/janpaul123","html_url":"https://github.com/janpaul123","followers_url":"https://api.github.com/users/janpaul123/followers","following_url":"https://api.github.com/users/janpaul123/following{/other_user}","gists_url":"https://api.github.com/users/janpaul123/gists{/gist_id}","starred_url":"https://api.github.com/users/janpaul123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/janpaul123/subscriptions","organizations_url":"https://api.github.com/users/janpaul123/orgs","repos_url":"https://api.github.com/users/janpaul123/repos","events_url":"https://api.github.com/users/janpaul123/events{/privacy}","received_events_url":"https://api.github.com/users/janpaul123/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-04-26T20:11:42Z","updated_at":"2015-04-26T20:38:25Z","closed_at":null,"body":"The HTML spec [doesn't require a `<head>` tag to be present](http://webdesign.about.com/od/beginningtutorials/qt/html-quick-and-dirty.htm). But your insertion algorithm [does assume a head to be present](https://github.com/ikreymer/pywb/blob/b379d15066445c58ff65d7f88018b51625d125ec/pywb/rewrite/rewrite_content.py#L237).\r\n\r\nYou can see this break in this example: https://via.hypothes.is/http://webdesign.about.com/od/examples/l/z-html4-minimal-document.htm\r\n\r\nNote that this is not completely hypothetical. At Factlink we ran into this problem with real webpages, though I have to admit I don't have an example handy right now. Our algorithm solves for this and proved pretty reliable: https://github.com/Factlink/web-proxy/blob/b24c5af013583311eeb7a9370cae652a30cc3eea/lib/html_editor.rb\r\n\r\nIt also prevents from accidentally injecting into a comment, like `<!-- some talk about <head> blah blah -->`."},"comment":{"url":"https://api.github.com/repos/hypothesis/via/issues/comments/96431854","html_url":"https://github.com/hypothesis/via/issues/9#issuecomment-96431854","issue_url":"https://api.github.com/repos/hypothesis/via/issues/9","id":96431854,"user":{"login":"ikreymer","id":1015759,"avatar_url":"https://avatars.githubusercontent.com/u/1015759?v=3","gravatar_id":"","url":"https://api.github.com/users/ikreymer","html_url":"https://github.com/ikreymer","followers_url":"https://api.github.com/users/ikreymer/followers","following_url":"https://api.github.com/users/ikreymer/following{/other_user}","gists_url":"https://api.github.com/users/ikreymer/gists{/gist_id}","starred_url":"https://api.github.com/users/ikreymer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ikreymer/subscriptions","organizations_url":"https://api.github.com/users/ikreymer/orgs","repos_url":"https://api.github.com/users/ikreymer/repos","events_url":"https://api.github.com/users/ikreymer/events{/privacy}","received_events_url":"https://api.github.com/users/ikreymer/received_events","type":"User","site_admin":false},"created_at":"2015-04-26T20:38:25Z","updated_at":"2015-04-26T20:38:25Z","body":"Hi, thanks for pointing this out, although this happens in a much more limited case than all pages w/o head tags.\r\n\r\nThe execution path you point out is actually a special banner-insert only mode, which skips the html parsing and attempts to insert a head through regex. (Currently only used for proxy mode I believe).\r\n\r\nThe general path goes through the HTMLRewriter, which has this:\r\nhttps://github.com/ikreymer/pywb/blob/master/pywb/rewrite/html_rewriter.py#L181\r\n\r\nIt takes a similar approach in detecting the first non-head tag, https://github.com/ikreymer/pywb/blob/master/pywb/rewrite/html_rewriter.py#L70\r\n\r\nSo if the page has any non-head tags, things should work correctly. But of course, as your example points out, its possible that a page only has head tags like title and no other tag. This seems much less likely to actually occur though, but should be fixed.\r\n\r\nIn such a case, I think, the only option is to add the head insert at the end, when the parser is being closed, if it hasn't been inserted already."}}