@nprofile1q... @nprofile1q... @nprofile1q... also FYI as someone with a modest understanding of the CS side of things: A few points:
1. This article is not trustworthy. It conflates very different types of AI system to maximize the threat it states, and that kind of dishonest reporting makes me distrust all of its conclusions. Expert diagnosis systems that use machine learning are a completely different kind of beast from a chatbot, even though that also uses machine learning in a different way. Touting the accuracy of specialized diagnosis systems as evidence for the effectiveness of chatbots is like showing off the excavating capability of a Volkswagen excavator add evidence that a BMW racing car will be faster than the competition.
2. Evaluating machine learning systems is very tricky, and it's not hard to get good-looking results in front of press if you have dollars to burn, which don't actually hold up in the real world. For the curious here's a nice thorough paper on issues with medical imaging specifically:
https://www.nature.com/articles/s41746-022-00592-yAs a patient, unless you show me in-the-field accuracy numbers from multiple years of deployment that actually rival human performance for a very particular task that this discussing AI is designed to do alone, I'd always rather have a human doctor's opinion than that of an AI system, and even when I'm willing to accept the AI diagnosis, I'd like a human doctor's second opinion and interpretation.
3. For medical-records and patient-interaction applications, including most of the ones listed in this article that chatbots are "good" at, I think doctors/hospitals using them are opening themselves up to a lot of liability and making a mistake. These systems make egregious errors in predictable patterns, which competent human staff do not make. Incompetent staff sometimes make the same errors, but the difference is that they are responsible for their own errors. If your "bedside manners" chatbot that takes over when the doctor is busy with another patient encourages a patient to kill themselves or take the wrong medicine, is that acceptable, since 99% of the time it speaks with a very reassuring manner? I guarantee you ChatGPT will make these mistakes orders of magnitude more often than even the most sleep-deprived RN or resident. We have seen these events happen already in other contexts; the medical/hospital context has many more opportunities for these failures. There is no real technological mitigation for them in the horizon either. Even for "transcribe my patient notes" I wouldn't trust dictation software unless needed as an accommodation, even though it has admittedly gotten pretty good. There's lots of opportunity for a missed jargon word to cause havoc in notes that get shared with someone else, for example.
4. The big AI companies have developed one tool that shows up very well I demos, but which has nasty flaws that make it unsuitable for a lot of what they're pushing. They are trying to sell their stuff as "the future" and say it must be "integrated" everywhere. If only you integrated our marvelous technology, your problems would be solved! This is backwards. A true solution looks first at the problem, and then asks "what tool would be best to use here?" By putting the tool choice first, you end up with ineffective or even counterproductive "solutions." This only makes sense if your goal is to sell the tools. For example, I understand doctors have very limited time but must write up notes between patient visits to refer to in future meetings. Sometimes notes end up inaccurate or illegible, or just muddled. How can we best solve this problem? Simple: hire more doctors to give them all more time to write better notes. Any alternate solution needs to be understood as a compromise. Might there be a way to use technology to help? Sure, there are probably plenty. Let's consider using a system to record the visit, then produce notes automatically by statistically predicting what notes the doctor would write. We'll have the doctor check them off every time. Is this good? No, because in this design, doctors will get lazy in their checking over time, especially if the system is very accurate for most visits. But such systems are going to make big mistakes for unusual visits, which doctors might not then correct. Even worse, by denying the doctor the cognitive task of organizing their thoughts into writing, you're disrupting the doctor's memory formation and chances to see unusual patterns or slight irregularities. Lowering the doctor's cognitive burden takes away the benefits you get from expending cognitive resources on the problem! As an alternate design, what if you had the doctor write the notes unaided, and then had a system try to flag possible discrepancies, misspellings, or illegible writing? Such a system might still be bad, if it creates too much friction (remember Clippy?). But it *might* be good if tuned correctly. It's not a flashy or "revolutionary" as the "we'll take your work away" system, but it avoids some of that system's worst drawbacks. There's probably even better designs I'm not thinking of. My point is that starting with "let's integrate a chatbot" is the wrong approach, and anyone who insists on it is not someone you should trust, because self-evidently they are stating from their own interests (sell/promote chatbots) while completely disregarding yours. They're basically saying "Can you help me think of a way to sell my product to you?" which is downright disrespectful.
Okay that's probably enough ranting from me. TL;DR: trust AI chatbots about as much as the least trustworthy intern you can imagine working with, because it will eventually make exactly the same kinds of disastrous mistakes, and you'll be the one to blame since (when we're not spinning yarns about machine intelligence) it's "just a tool that you chose to use." It's not even capable of learning from those mistakes either, because to let chstbots learn from their public interactions would be dangerous in other ways (see Microsoft Tay).
