One fetcher bot seen smacking a website with 39,000 requests per minute
Cloud services giant Fastly has released a report claiming AI crawlers are putting a heavy load on the open web, slurping up sites at a rate that accounts for 80 percent of all AI bot traffic, with the remaining 20 percent used by AI fetchers. Bots and fetchers can hit websites hard, demanding data from a single site in thousands of requests per minute.
Source Link: https://educronix.com/ai-crawlers-and-fetchers-are-blowing-up-websites-with-meta-and-openai-the-worst-offenders/
Author: Ernestro Casas -
Published on: