A new report from Cloudflare accuses Perplexity of ignoring robots.txt files and concealing the identity of its AI crawlers to get past websites’ restrictions.
The AI search startup Perplexity is allegedly skirting restrictions meant to stop its AI web crawlers from accessing certain websites, according to a report from Cloudflare. In the report, Cloudflare claims that when Perplexity encounters a block, the startup will conceal its crawling identity “in an attempt to circumvent the website’s preferences.
” But if the website has restrictions against AI scraping, Cloudflare claims Perplexity will change its user agent — the bit of information that tells a website what kind of browser and device you’re using, or if the visitor is a bot — to “impersonate Google Chrome on macOS.” Cloudflare says this “undeclared crawler” uses “rotating” IP addresses that the company doesn’t include on the list of IP addresses used by its bots.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Coinbase (COIN) Partners With Perplexity AI to Bring Real-Time Crypto Market Data to TradersThe tie-up will allow users to dig into market trends, monitor price action and explore token fundamentals.
Read more »
Perplexity launches AI-powered web browser for select group of subscribersPerplexity AI launched a new artificial intelligence-powered web browser called Comet
Read more »
Perplexity just launched an AI web browserPerplexity has launched Comet, a web browser that uses Perplexity as its default search engine and comes with a built-in AI assistant.
Read more »
Perplexity launches AI-powered Comet browser, and there’s a big reason to stay awayPerplexity launched its own browser called Comet, and I have a big reason to avoid it. Spoiler, it’s not the price.
Read more »
Perplexity CEO says AI tools cut development time from days to hoursBusiness Insider tells the global tech, finance, stock market, media, economy, lifestyle, real estate, AI and innovative stories you want to know.
Read more »
Perplexity’s CEO on why the browser is AI’s killer appAravind Srinivas discusses Perplexity’s new Comet browser and the future of the web.
Read more »