Running your own grasp server is the right next step. Your relay set stays part of the repo's identity though: clone URLs live in the repo announcement, and patches and ref updates travel as events. I would put more than one clone endpoint in that announcement, so clients can fail over when a server is down. Relays never carry your packfiles, they only coordinate them.