Are they getting the geolocation from your Sim maybe? Your Sim can be used to trace you via cell towers and isn't affected by a VPN.
How can it be called unprivileged if it has access to SIM geolocation, however that works