if your website is hidden behind a special dns resolver does it prevent ai scrapers
I've been mulling over how to publish writing in 2026 without being mixed into the soul of claude code
@Tlapkaš¾ :neocat_phone: if they can't be bothered to write a scraper that clones git repos instead of hammering diff pages with requests then I imagine they couldn't be bothered to make one that uses a custom DNS resolver either. Then again the amount of effort they put into evading blocksā¦
In theory if your reverse proxy doesn't serve anything unless they have the correct host header or SNI then they wouldn't be able to scrape it via IP address.
In theory if your reverse proxy doesn't serve anything unless they have the correct host header or SNI then they wouldn't be able to scrape it via IP address.