I've already banned a few IPs, I'll consider this if it continues to be a problem
Good luck!
I don’t know how things are right now, but when I did this last year there were simply too many crawlers to even try to block manually. Plus, they don’t play fair. Lots of scrapers were spoofing IPs and using a multitude of IPv4 and IPv6 addresses. Even my firewall on a 128 GB memory server started to struggle to keep up. Anubis helped a bit, but things were still quite painful. I did this before Cloudflare enhanced its bot protection features. Assuming analytics can be trusted, China and the US alone were sending more traffic to my Forgejo instance, which only had a bunch of empty projects, than what I typically get on Nostr, ActivityPub, and IRC combined in a month.
Keeping my internal repo and syncing to public forges like GitHub, GitLab, and Codeberg is a good enough compromise for my needs while GRASP (and other non Nostr alternatives) matures. It also helps with the problem of nobody wanting to report issues or contribute outside of popular forges.
I don’t know how things are right now, but when I did this last year there were simply too many crawlers to even try to block manually. Plus, they don’t play fair. Lots of scrapers were spoofing IPs and using a multitude of IPv4 and IPv6 addresses. Even my firewall on a 128 GB memory server started to struggle to keep up. Anubis helped a bit, but things were still quite painful. I did this before Cloudflare enhanced its bot protection features. Assuming analytics can be trusted, China and the US alone were sending more traffic to my Forgejo instance, which only had a bunch of empty projects, than what I typically get on Nostr, ActivityPub, and IRC combined in a month.
Keeping my internal repo and syncing to public forges like GitHub, GitLab, and Codeberg is a good enough compromise for my needs while GRASP (and other non Nostr alternatives) matures. It also helps with the problem of nobody wanting to report issues or contribute outside of popular forges.
1❤️1