{"action":"created","issue":{"url":"https://api.github.com/repos/IQSS/dataverse/issues/2274","labels_url":"https://api.github.com/repos/IQSS/dataverse/issues/2274/labels{/name}","comments_url":"https://api.github.com/repos/IQSS/dataverse/issues/2274/comments","events_url":"https://api.github.com/repos/IQSS/dataverse/issues/2274/events","html_url":"https://github.com/IQSS/dataverse/issues/2274","id":88809206,"number":2274,"title":"Robots txt: Enable Google","user":{"login":"eaquigley","id":5922904,"avatar_url":"https://avatars.githubusercontent.com/u/5922904?v=3","gravatar_id":"","url":"https://api.github.com/users/eaquigley","html_url":"https://github.com/eaquigley","followers_url":"https://api.github.com/users/eaquigley/followers","following_url":"https://api.github.com/users/eaquigley/following{/other_user}","gists_url":"https://api.github.com/users/eaquigley/gists{/gist_id}","starred_url":"https://api.github.com/users/eaquigley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eaquigley/subscriptions","organizations_url":"https://api.github.com/users/eaquigley/orgs","repos_url":"https://api.github.com/users/eaquigley/repos","events_url":"https://api.github.com/users/eaquigley/events{/privacy}","received_events_url":"https://api.github.com/users/eaquigley/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/IQSS/dataverse/labels/Status:%20QA","name":"Status: QA","color":"336633"}],"state":"open","locked":false,"assignee":{"login":"scolapasta","id":4268292,"avatar_url":"https://avatars.githubusercontent.com/u/4268292?v=3","gravatar_id":"","url":"https://api.github.com/users/scolapasta","html_url":"https://github.com/scolapasta","followers_url":"https://api.github.com/users/scolapasta/followers","following_url":"https://api.github.com/users/scolapasta/following{/other_user}","gists_url":"https://api.github.com/users/scolapasta/gists{/gist_id}","starred_url":"https://api.github.com/users/scolapasta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scolapasta/subscriptions","organizations_url":"https://api.github.com/users/scolapasta/orgs","repos_url":"https://api.github.com/users/scolapasta/repos","events_url":"https://api.github.com/users/scolapasta/events{/privacy}","received_events_url":"https://api.github.com/users/scolapasta/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/IQSS/dataverse/milestones/38","html_url":"https://github.com/IQSS/dataverse/milestones/4.0.2","labels_url":"https://api.github.com/repos/IQSS/dataverse/milestones/38/labels","id":1107604,"number":38,"title":"4.0.2","description":"Working on:\r\n\r\n- Data Related to Me\r\n- Linked dataverses and datasets","creator":{"login":"mcrosas","id":676040,"avatar_url":"https://avatars.githubusercontent.com/u/676040?v=3","gravatar_id":"","url":"https://api.github.com/users/mcrosas","html_url":"https://github.com/mcrosas","followers_url":"https://api.github.com/users/mcrosas/followers","following_url":"https://api.github.com/users/mcrosas/following{/other_user}","gists_url":"https://api.github.com/users/mcrosas/gists{/gist_id}","starred_url":"https://api.github.com/users/mcrosas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcrosas/subscriptions","organizations_url":"https://api.github.com/users/mcrosas/orgs","repos_url":"https://api.github.com/users/mcrosas/repos","events_url":"https://api.github.com/users/mcrosas/events{/privacy}","received_events_url":"https://api.github.com/users/mcrosas/received_events","type":"User","site_admin":false},"open_issues":40,"closed_issues":1,"state":"open","created_at":"2015-05-12T21:57:24Z","updated_at":"2015-06-16T19:16:14Z","due_on":"2015-07-08T04:00:00Z","closed_at":null},"comments":2,"created_at":"2015-06-16T18:54:14Z","updated_at":"2015-06-17T13:52:07Z","closed_at":null,"body":"Need to enable Google in the robots.txt file so datasets can be discovered in Google Scholar: https://dataverse.harvard.edu/robots.txt\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/IQSS/dataverse/issues/comments/112808917","html_url":"https://github.com/IQSS/dataverse/issues/2274#issuecomment-112808917","issue_url":"https://api.github.com/repos/IQSS/dataverse/issues/2274","id":112808917,"user":{"login":"pdurbin","id":21006,"avatar_url":"https://avatars.githubusercontent.com/u/21006?v=3","gravatar_id":"","url":"https://api.github.com/users/pdurbin","html_url":"https://github.com/pdurbin","followers_url":"https://api.github.com/users/pdurbin/followers","following_url":"https://api.github.com/users/pdurbin/following{/other_user}","gists_url":"https://api.github.com/users/pdurbin/gists{/gist_id}","starred_url":"https://api.github.com/users/pdurbin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pdurbin/subscriptions","organizations_url":"https://api.github.com/users/pdurbin/orgs","repos_url":"https://api.github.com/users/pdurbin/repos","events_url":"https://api.github.com/users/pdurbin/events{/privacy}","received_events_url":"https://api.github.com/users/pdurbin/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T13:52:07Z","updated_at":"2015-06-17T13:52:07Z","body":"I noticed https://github.com/IQSS/dataverse/blob/4.0.2/src/main/webapp/robots.txt was not changed so I think what's happening is that we want robots.txt to block Google for all the **test** instances of Dataverse. That is, Google is blocked by default. For a **production** instance I assume there's some procedure for letting Google crawl a Dataverse instance. Probably this should be documented somewhere."}}