{"action":"created","issue":{"url":"https://api.github.com/repos/hibernate/in.relation.to/issues/19","labels_url":"https://api.github.com/repos/hibernate/in.relation.to/issues/19/labels{/name}","comments_url":"https://api.github.com/repos/hibernate/in.relation.to/issues/19/comments","events_url":"https://api.github.com/repos/hibernate/in.relation.to/issues/19/events","html_url":"https://github.com/hibernate/in.relation.to/pull/19","id":88706259,"number":19,"title":"WEBSITE-265: Generate redirects for old URLs","user":{"login":"gunnarmorling","id":28612,"avatar_url":"https://avatars.githubusercontent.com/u/28612?v=3","gravatar_id":"","url":"https://api.github.com/users/gunnarmorling","html_url":"https://github.com/gunnarmorling","followers_url":"https://api.github.com/users/gunnarmorling/followers","following_url":"https://api.github.com/users/gunnarmorling/following{/other_user}","gists_url":"https://api.github.com/users/gunnarmorling/gists{/gist_id}","starred_url":"https://api.github.com/users/gunnarmorling/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gunnarmorling/subscriptions","organizations_url":"https://api.github.com/users/gunnarmorling/orgs","repos_url":"https://api.github.com/users/gunnarmorling/repos","events_url":"https://api.github.com/users/gunnarmorling/events{/privacy}","received_events_url":"https://api.github.com/users/gunnarmorling/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"Sanne","id":42573,"avatar_url":"https://avatars.githubusercontent.com/u/42573?v=3","gravatar_id":"","url":"https://api.github.com/users/Sanne","html_url":"https://github.com/Sanne","followers_url":"https://api.github.com/users/Sanne/followers","following_url":"https://api.github.com/users/Sanne/following{/other_user}","gists_url":"https://api.github.com/users/Sanne/gists{/gist_id}","starred_url":"https://api.github.com/users/Sanne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Sanne/subscriptions","organizations_url":"https://api.github.com/users/Sanne/orgs","repos_url":"https://api.github.com/users/Sanne/repos","events_url":"https://api.github.com/users/Sanne/events{/privacy}","received_events_url":"https://api.github.com/users/Sanne/received_events","type":"User","site_admin":false},"milestone":null,"comments":8,"created_at":"2015-06-16T11:59:37Z","updated_at":"2015-06-16T15:54:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/hibernate/in.relation.to/pulls/19","html_url":"https://github.com/hibernate/in.relation.to/pull/19","diff_url":"https://github.com/hibernate/in.relation.to/pull/19.diff","patch_url":"https://github.com/hibernate/in.relation.to/pull/19.patch"},"body":"This is largely done, so it's time to get some feedback. What has happened:\r\n\r\n* _importer/importer.rb has been extended to create a .htaccess fragment with redirects for actual posts and tags\r\n* _importer/user_redirect_creator.rb has been provided to generate a .htaccess fragment with redirects for user/blogger URLs\r\n* _importer/crawler.rb has been extended to parse several URLs, with a depth of n (so to quickly crawl a given set of sub-pages)\r\n\r\nThat's the work-flow:\r\n\r\n* Run the importer on the latest posts.pstore file (`bundle exec ./importer.rb -s ../posts-19-03-2015.pstore -o ../posts -rf ../redirects/.htaccess_posts -ni -na`): This will create the posts .htaccess fragment\r\n* Create a pstore file with the home pages of all users (only once, could be shared via DropBox):\r\n    bundle exec ./crawler.rb -u \"http://in.relation.to/Bloggers/Ales,http://in.relation.to/Bloggers/Aslak,http://in.relation.to/Bloggers/Brett,http://in.relation.to/Bloggers/Christian,http://in.relation.to/Bloggers/Dan,http://in.relation.to/Bloggers/Daniel,http://in.relation.to/Bloggers/David,http://in.relation.to/Bloggers/Davide,http://in.relation.to/Bloggers/Emmanuel,http://in.relation.to/Bloggers/Gail,http://in.relation.to/Bloggers/Gavin,http://in.relation.to/Bloggers/Gleb,http://in.relation.to/Bloggers/Greg,http://in.relation.to/Bloggers/Gunnar,http://in.relation.to/Bloggers/Hardy,http://in.relation.to/Bloggers/Ilya,http://in.relation.to/Bloggers/Jason,http://in.relation.to/Bloggers/JasonP,http://in.relation.to/Bloggers/Jay,http://in.relation.to/Bloggers/Jesper,http://in.relation.to/Bloggers/John,http://in.relation.to/Bloggers/Jozef,http://in.relation.to/Bloggers/Ken,http://in.relation.to/Bloggers/Lincoln,http://in.relation.to/Bloggers/Luksa,http://in.relation.to/Bloggers/Marek,http://in.relation.to/Bloggers/Marius,http://in.relation.to/Bloggers/Marko,http://in.relation.to/Bloggers/Martin,http://in.relation.to/Bloggers/Max,http://in.relation.to/Bloggers/Michael,http://in.relation.to/Bloggers/Nicklas,http://in.relation.to/Bloggers/Norman,http://in.relation.to/Bloggers/Pete,http://in.relation.to/Bloggers/Robb,http://in.relation.to/Bloggers/Sanne,http://in.relation.to/Bloggers/Scott,http://in.relation.to/Bloggers/Sergey,http://in.relation.to/Bloggers/Shane,http://in.relation.to/Bloggers/Snjezana,http://in.relation.to/Bloggers/Steve,http://in.relation.to/Bloggers/StrongLiu,http://in.relation.to/Bloggers/Stuart,http://in.relation.to/Bloggers/Tihomir\" -p \".*\" -o users.pstore -d 0\r\n* Create the blogger .htaccess fragement: `bundle exec ./user_redirect_creator.rb -s users.pstore -o ../redirects/.htaccess_bloggers`\r\n* Merge the fragments to a combined .htaccess file: `cat ../redirects/.htaccess_misc ../redirects/.htaccess_bloggers ../redirects/.htaccess_posts > ../_site/.htaccess`\r\n\r\nOpen questions:\r\n\r\n* Are all (important) URLs covered? What is missing (manual additions can go into .htaccess_misc)?\r\n* Casing of posts; I saw Emmanuel and Hardy discuss about this yesterday. If casing differs between old and new URLs, that'd have to be considered\r\n* What to do about bloggers with 0 posts? E.g. http://in.relation.to/Bloggers/Snjezana has no posts at all; I think we can ignore those from the beginning?\r\n* Is there a user profile page already? I.e. a counter-part to e.g. http://in.relation.to/user/gunnar, or should these just redirect to the user's posts?\r\n* Instead of .htaccess the redirects should go into the main Apache config for perf reasons; .htaccess is quicker for development though as changes get instantly visible\r\n* Currently headers are sent with 301 responses to suppress browser caching of redirects; That's good for now as otherwise it's nearly impossible to fix wrong re-directs (users would have to empty browser caches); But in the long run, if the redirects prove good, we could remove these headers and enable caching\r\n* Just coming to my mind, what about attachments? Is it worth to create redirects for those as well?\r\n* Need to incorporate tag aggregation rules created as part of WEBSITE-299"},"comment":{"url":"https://api.github.com/repos/hibernate/in.relation.to/issues/comments/112478145","html_url":"https://github.com/hibernate/in.relation.to/pull/19#issuecomment-112478145","issue_url":"https://api.github.com/repos/hibernate/in.relation.to/issues/19","id":112478145,"user":{"login":"emmanuelbernard","id":300760,"avatar_url":"https://avatars.githubusercontent.com/u/300760?v=3","gravatar_id":"","url":"https://api.github.com/users/emmanuelbernard","html_url":"https://github.com/emmanuelbernard","followers_url":"https://api.github.com/users/emmanuelbernard/followers","following_url":"https://api.github.com/users/emmanuelbernard/following{/other_user}","gists_url":"https://api.github.com/users/emmanuelbernard/gists{/gist_id}","starred_url":"https://api.github.com/users/emmanuelbernard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emmanuelbernard/subscriptions","organizations_url":"https://api.github.com/users/emmanuelbernard/orgs","repos_url":"https://api.github.com/users/emmanuelbernard/repos","events_url":"https://api.github.com/users/emmanuelbernard/events{/privacy}","received_events_url":"https://api.github.com/users/emmanuelbernard/received_events","type":"User","site_admin":false},"created_at":"2015-06-16T15:54:32Z","updated_at":"2015-06-16T15:54:32Z","body":"Gunnar can you share the .htaccess or .htaccess files, with my super slow connection it is not practical to  run the importer"}}