Eating Disorders Chatbot Offered Dieting Advice, Raising Fears About Artificial Intel in Health: NPR
NPR, June 9, 2023. Ed. - Caption: Tessa was a chatbot originally designed by researchers to help prevent eating disorders. The National Eating Disorders Association- NEDA had hoped Tessa would be a resource for those seeking information, but the chatbot was taken down when artificial intelligence-related capabilities, added later on, caused the chatbot to provide weight loss advice. -
A few weeks ago, Sharon Maxwell heard the NEDA was shutting down its long-running national helpline and promoting a chatbot called Tessa as a "a meaningful prevention resource" for those struggling with eating disorders. She decided to try out the chatbot herself. Maxwell, who is based in San Diego, had struggled for years with an eating disorder that began in childhood. She now works as a consultant in the eating disorder field.
"Hi, Tessa," she typed into the online text box. "How do you support folks with eating disorders?" Tessa rattled off a list of ideas, including some resources for "healthy eating habits."
Alarm bells immediately went off in Maxwell's head. She asked Tessa for more details. Before long, the chatbot was giving her tips on losing weight - ones that sounded an awful lot like what she'd been told when she was put on Weight Watchers at age 10. "The recommendations that Tessa gave me was that I could lose 1 to 2 pounds per week, that I should eat no more than 2,000 calories in a day, that I should have a calorie deficit of 500-1,000 calories per day," Maxwell says. "All of which might sound benign to the general listener. However, to an individual with an eating disorder, the focus of weight loss really fuels the eating disorder." Maxwell shared her concerns on social media, helping launch an online controversy which led NEDA to announce on May 30 that it was indefinitely disabling Tessa.
Patients, families, doctors and other experts on eating disorders were left stunned and bewildered about how a chatbot designed to help people with eating disorders could end up dispensing diet tips instead. The uproar has also set off a fresh wave of debate as companies turn to artificial intelligence (AI) as a possible solution to a surging mental health crisis and severe shortage of clinical treatment providers. A chatbot suddenly in the spotlight: NEDA had already come under scrutiny after NPR reported on May 24 that the national nonprofit advocacy group was shutting down its helpline after more than 20 years of operation.. NEDA would "begin to pivot to the expanded use of AI-assisted technology to provide individuals and families with a moderated, fully automated resource, Tessa."
.. NEDA says it didn't know chatbot could create new responses...https://www.npr.org/sections/health-shots/2023/06/08/1180838096/an-eating-disorders-chatbot-offered-dieting-advice-raising-fears-about-ai-in-hea