this post was submitted on 09 Jul 2025
95 points (98.0% liked)

Technology

72606 readers
3426 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
top 15 comments
sorted by: hot top controversial new old
[–] Pika@sh.itjust.works 21 points 17 hours ago (1 children)

I mean, with a company as large as cloudflare. I think they could /easily/ strong-arm this move by making blocking google crawlers a default setting on websites. The amount of traffic drop alone from that would make google think twice about the whole ordeal. And people who care about the google search indexers can turn them on again which will allow indexing again. but a default block would cause a lot of disruption google side and many people I don't think would go in and fix the setting till later on down the road.

[–] sbv@sh.itjust.works 15 points 16 hours ago (2 children)

Cloudflare's customers probably wouldn't be on board with that. Google's properties provide a tonne of traffic to businesses. Doing anything to put that in jeopardy would probably have many of Cloudflare's customers looking for a new provider.

[–] Glitchvid@lemmy.world 10 points 15 hours ago (1 children)

Google used to provide a ton of traffic, they hoard it all themselves now through AI and summaries of content. Eventually the balance of cost/benefit will shift and Google will suddenly see itself rejected from scraping, furthering the product deathspiral.

[–] pupbiru@aussie.zone 4 points 9 hours ago (1 children)

content is only 1 category of website

ecommerce drives all the advertising that funds content… it’s a much bigger market, and they don’t care about content scraping as long as you buy their product

[–] Glitchvid@lemmy.world 2 points 8 hours ago

And the long term plan there is to strangle sites and take %100 of the adrev spend for themselves since users won't ever leave the Google site. Either way Google as a search engine enters a death spiral, it's already bleeding users.

[–] Pika@sh.itjust.works 4 points 16 hours ago (1 children)

it would need to be advertised as a change and have it as a setting that had to be set yea, just have it default blocking abusive trackers, having Google bot or whatever it's crawler name is as on there, with a toggle to allow it again

[–] osaerisxero@kbin.melroy.org 2 points 16 hours ago

Alternatively, you use the cloudflare money to sue the monopoly to decouple search and all other products, since blocking the AI trawlers shouldn't have any measurable impact in search rankings

[–] fubarx@lemmy.world 9 points 14 hours ago (2 children)

Totally understandable.

If scanning to help send traffic to your website, that's cool. If scanning to generate summaries that won't send any traffic your way. No bueno.

Ultimately, it should be whatever most benefits users.

[–] acosmichippo@lemmy.world 4 points 13 hours ago* (last edited 13 hours ago)

but there also needs to be incentive for sites to host content. if it all gets hijacked by search engines that isn't sustainable.

[–] Outwit1294@lemmy.today 3 points 10 hours ago

No, things should not benefit users, they should benefit the creator of the original content.

[–] Outwit1294@lemmy.today 8 points 10 hours ago (1 children)

If I had to choose my favourite corporation, it would be Cloudflare. They at least do something good.

[–] r00ty@kbin.life 3 points 7 hours ago

There seems to be a line, so far as I can tell. If everything you need sits on the free tier, they're really good (well tbh their R2 storage is reasonably priced too). But once you stray into needing a paid tier, it apparently (I'm not there) quickly gets expensive as you're lured into every higher tiers.

But yes, in general I don't mind cloudflare so much and do use their free (and R2 paid) services.

[–] BaroqueInMind@piefed.social 4 points 17 hours ago* (last edited 17 hours ago)

Additionally, Cloudflare's initiative faces criticism from those who "worry that academic research, security scans, and other types of benign web crawling will get elbowed out of websites as barriers are built around more sites" through Cloudflare's blocks and paywalls, the WSJ reported.

The fuck? Since when is a bot designed to enumerate your network weaknesses to sell to Russian/Chinese/US hacking groups a bad thing to block? Fuck the WSJ for even putting that dumb as fuck take on the internet for other idiots to think about.

NO , its not a good fucking idea to allow the equivalent of an incessant door-to-door salesman into your home to take notes of everything you own and sell to a random motherfucker somewhere else you don't know.

That behavior is fucking weird and shouldn't be tolerated. Cloudflare arbitrarily blocking that network traffic for you is a good thing.

[–] Zwuzelmaus@feddit.org 4 points 11 hours ago

Google has existed long enough now. The are allowed to disappear.

[–] 2xsaiko@discuss.tchncs.de 1 points 17 hours ago

Google’s bot is fine in my book, their crawler doesn’t absolutely blast your server with web requests like other AI crawlers do. (Speaking of, I need to update my list of netblocks and UAs to get iocaine-holed.)

That said, two evil megacorps potentially fighting? I hope they kill each other.