 On July 1, 2019 Google announced the proposal of the Robots Exclusion Protocol as an official standard under Internet Engineering Task Force.The draft will now undergo acceptance process.When a site owner wishes to give instructions to web robots they place a text file called in the root of the web site hierarchy (e.g.