Shouldn’t you have redundant TEEs that, using remote attestation, copy the keys so that you have a backup?
Bill Gates' dream of total control, enforced by Intel's SGX, is this your idea of security?