If you store the same private key on both a Trezor and a Ledger, you expose it to vulnerabilities on both stacks. So Ledger using a closed source "Secure Element" might have given you an insecure priv...
It depends on where you see risk. I view manufacturer failure, hardware failure, lost wallet, as much higher risks than private key breach.
1
