{"push_id":541148207,"size":1,"distinct_size":0,"ref":"refs/heads/master","head":"21b81462924aa53236e9616ebcaf1af02b76ebb5","before":"39c972cc948980acd0abc11eb4683638e7c6623b","commits":[{"sha":"21b81462924aa53236e9616ebcaf1af02b76ebb5","author":{"email":"fc25e4ac8e69f43482550bdb547940a4ca80c3f1@exo.org.uk","name":"Struan Donald"},"message":"Allow instances to set robots.txt to Disallow all\n\nAdd a no spider option to the about page that when enabled serves up a\nrobots.txt file that disallows all. If not set then serves up the normal\nrobots.txt\n\nAlso adds a lib to serve up robots.txt which means moving the file\nto views as otherwise the static serving kicks in first and we need to\nprocess this later on in order to get access to the instance settings.\n\nThis does also have the side effect that we have separate robots.txt\nfiles for the hosting and instance apps.\n\nFixes #692","distinct":false,"url":"https://api.github.com/repos/mysociety/popit/commits/21b81462924aa53236e9616ebcaf1af02b76ebb5"}]}