IMO, there's no need to throw the baby out with the bathwater. Just don't rely on RNG for seed generation. Verify that your coldcard is processing dice rolls correctly (perhaps with several test cases); you should be fine.
It shouldn't be surprising that if something was going to fall through the cracks (on any signing device), it would be reliance on RNG. It's one of the few functions that isn't deterministic, and therefore can't be independently verified is working properly. So, just don't rely on RNG for seed generation, period.
https://coldcard.com/docs/verifying-dice-roll-math/