Attack Of The AI Crawlers
ComputerWorld, Monday, May 5th, 2025
Unlike their search engine counterparts, scraper bots in search of content for genAI model training often go where site owners tell them not to go. Some sites are getting hit millions of times and incurring massive bandwidth charges. What can IT do?
Enterprise IT leaders - and their counterparts in Legal and Compliance - have many reasons to hate having their websites visited by genAI model makers' agents, whether they are called bots, crawlers, or spiders.
They could object to their IP being stolen and used to train genAI models with almost no benefit to their company. They might be infuriated by copyright and trademark violations and the exposure of their customers' and employees' personally identifiable information to the world of thieves.