Damus
Open in Damus
LisPi · 65w
Misbehaving scrappers overwhelming servers is just a demonstration of yet another fatal flaw in the server-centric model of networked communications. The solution isn't to make it worse for humans (a...
 truh profile picture
truh @truh
@LisPi At least half of the complaints I see people make where specific services are mentioned are about scrapers scraping git forges. There is a very straightforward way to get the complete content of the repo (git clone) and the scrapers still try to request a webview of every single revision of every single file and tar balls for every single revision on top.

Wikipedia is also getting flooded with AI scraper traffic.
1
LisPi · 65w
nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqfv3fz4q3y9xeyp9cxn92thp9wc7llfkg5jucnvaaffl5y3m9t02s6fyv8y There is no good option besides rate-limiting unauthenticated accesses for interactions that cannot be backed solely by content-addressed mechanisms. The tarball would be a single linea...