 It is used when the multiple burst of visits from bots is slowing down the host.Yandex interprets the value as the number of seconds to wait between subsequent visits.Bing defines crawl-delay as the size of a time window (from 1 to 30 seconds) during which BingBot will access a web site only once.