Ever considered using Frost to aggregate the device keypairs (with the added benefit of enrolling new device keys, or removing old device keys, later while keeping the same aggregate pubkey)?

The user encryption definitely needs to be decoupled.