I agree, I can't wait for real people to be replaced by competent AI. But it seems there has been like no progress in improving these agents

Because the current iterations are not AIs. Most are nothing more than statically programmed workflows. There are some companies out there that are now offering LLM style voice agents, but adoption is low due to the cost.