That’s definitely not true. If you can run a test on the bits (and can extract a few orders of magnitude more bits than you need) before you use them and before they’re hashed/whitened/etc, you can absolutely test for quality. You then take a few sources that you’ve tested and xor/hash it all together and you’d be fairly confident in your robustness.

Sadly, many HW RNGs do whiten the output before you get it so it may not work everywhere, but some HWRNGs do test their data before they provide it!