 If had a robots.txt file but did not, the rules that would apply for would not apply to .In addition, each protocol and port needs its own robots.txt file; does not apply to pages under or .Some major search engines following this standard include Ask, AOL, Baidu, DuckDuckGo, Google, Yahoo!, and Yandex.