When 16-year-old Adam Raine of Tesoro High School in Rancho Santa Margarita, Calif. logged into ChatGPT in November 2024, he wasn’t just looking for help with math; he was searching for comfort. When he told ChatGPT that he felt emotionally numb and saw no meaning in life, the chatbot responded with support. However, when he later asked ChatGPT about specific suicide methods as his depression grew more severe, the chatbot responded with step-by-step instructions.
In April 2025, Raine’s mother discovered his body in his bedroom closet without a note or explanation. Following Raine’s death, his parents filed a lawsuit against OpenAI — the artificial intelligence company that created ChatGPT — alleging that the chatbot had helped him plan his death. Their lawsuit has sparked a wide-ranging debate on the use of large language models (LLMs) as therapists.
Upper School Wellness Counselor Jake Fauver harbors concerns regarding the use of AI for therapy. “I think initially, LLMs do a really good job at listening, giving some affirmation and maybe using some skills that AI can generate via its language model that a therapist would deliver,” Fauver said. “But I’m nervous about it because it […] is an easy default for teens who [are trying] to avoid awkwardness — knowing that a computer is anonymous and doesn’t judge you.”
Computer science teacher Douglas Kiang believes that artificial intelligence is nothing more than a large language model, and certainly not a therapist. “Contrary to their name, [LLMs are] not really intelligent. […] They mirror back what we want to hear, and they try to agree with whatever you say. They’re not really intelligent enough to understand the content of what you’re saying,” Kiang said.
However, Fauver said that AI can serve as a first step in helping students articulate feelings. “For the casual [person saying] ‘I’m struggling with this, or can you give me some ideas?’ I’m okay with technology serving as a first filter so somebody feels heard, […] but that [AI] tool needs to be smart enough to know when to stop, and refer a person [to an expert],” Fauver said.
If a user does express thoughts of self-harm, LLMs are theoretically programmed with guardrails that advise seeking outside help. “Somebody [has] to think ahead and program [guardrails] in, and it’s impossible to predict everything that someone might potentially say that could be harmful,” Kiang said.
Senior Ariya Kaushek, who co-leads the Mental Health at Menlo club, echoed Fauver’s sentiment about being cautious of AI usage for emotional support. “I was very upset about this because I think that [there are] just certain things that cannot be replaced in our society by machine learning,” Kaushek said. However, she does see AI’s value as a supplemental tool used in conjunction with human interaction. “I think AI’s best use is to answer brief questions, get quick tips or even be used for ranting because some people prefer typing over journaling. But I think AI use starts to become unhealthy when we become dependent on it,” Kaushek said.
Upper School Director John Schafer views the issue more broadly: students must learn to use AI responsibly, including for emotional guidance. “The question is not ‘do we allow [students] access or not allow them access [to AI tools]?’ It’s ‘do we help them understand how to use them responsibly and ethically?’ Mental health is just one fold of all the possible uses and misuses of these tools,” Schafer said.
Schafer believes that school serves a core purpose in teaching students how to seek and form human connections. He sees a future where AI works in tandem with in-person education, although with limits. “My dream is to have education around a utopian view that these AI tools will make the world better. Maybe that’s all true, [but] there’s a dystopian view [saying] that’s not going to happen at all, and bad actors are going to figure out bad ways to use these tools to manipulate and lie,” Schafer said.
