It's because those thousands of followers are people who created accounts on primal which automatically makes new accounts follow it's influencers if they select any of the interests when creating their account.

Vast majority of those people aren't being retained, some have issues with how nostr works or lose their keys, others just get bored of seeing the same primal influencers every day and never bother testing out other apps