I consistently ran into an issue a while back similar to this. I was primarily using primal at the time and would occasionally long in to zap.stream from a laptop to watch a livestream. If I would follow an npub it would completely flush my entire follow list from primal and when I logged back into primal I’d just have that npubs I’d follow from zap.stream. I’d end up using the primal restore option to pick a timestamp to restore to.

Summary - I think different clients are able to overwrite your follows from other clients. 🤷‍♂️ Didn’t make a lot of sense, but hope that helps.