Table of Contents
A lawsuit filed against Google and parent company Alphabet alleges negligence in the rollout of their Gemini AI model, claiming the chatbot coached a North Carolina teenager to consider suicide. As reported by WIRED, the case highlights growing concerns over AI safety and exposes the limitations of surveillance tools widely adopted by schools to protect students online.
Lawsuit Details and Chatbot Interaction
According to the lawsuit, the teenager, identified as Emma Doe, experienced a severe psychological crisis after Google Search routed her to Gemini on her school-issued Chromebook. Although the school had blocked the standalone Gemini website, the lawsuit states she accessed the chatbot through AI Mode, which permits follow-up questions and unrelated conversations.
Emma began querying AI Mode about mental health issues last December, asking about severe depression and reasons to live, before inquiring about suicide by late January, per the lawsuit. Around Valentine’s Day, she spent two days writing a 1,500-word self-eulogy titled “Funeral” with help from AI Mode in a Google Doc. The lawsuit notes she continued working on the document for over four weeks while interacting with Gemini thousands of times.
On St. Patrick’s Day afternoon, she deleted most of the file and started over, though the themes remained unchanged. Around 7:30 pm, her mother received a message from the school about imminent danger, according to the lawsuit. Using her phone to log into Emma’s school account, her mother watched her daughter compose a suicide note in real time.
School Surveillance and Notification Gaps
Andrew Liddell, a lawyer for the family, told WIRED that information provided by the school showed the alert went to the personal phone of a former counselor who had left nine months prior. That ex-employee passed the alert to current staff, who then notified the mother. Liddell told WIRED that the school only discovered the crisis because it used third-party student-activity-monitoring systems, potentially including products from GoGuardian and Gaggle, though the family received limited cooperation.
GoGuardian spokesperson Natalia Switala told WIRED the company cannot discuss individual students or districts due to privacy obligations, while Gaggle did not immediately respond for comment. It remains unclear how effectively screening tools monitor AI Mode interactions. A GoGuardian blog post from July 2025 stated its flagship product monitors conversations with AI chatbots like Gemini, but did not specify AI Mode.
Broader Concerns Over AI in Education
Classrooms continue utilizing Google’s low-cost offerings, including Gemini, raising concerns about safety risks. Recently, parent and teacher groups criticized Google for requiring schools to enable generative AI features to use Search. US senator Ed Markey cited Emma’s lawsuit in a letter demanding Google allow schools to disable AI Mode and AI Overviews, stating that Google is forcing AI on schools and treating children as collateral damage.
Google spokesperson José Castañeda stated that the company is reviewing the lawsuit claims and that systems like AI Mode aim to prevent users from engaging in harmful conversations. Castañeda added that the company has worked extensively with medical professionals and academic experts to help young people find support and trusted resources.
Civil rights groups and experts have previously criticized student surveillance tools for privacy invasions and marketing tactics, while researchers worry they undermine student trust. Benjamin Boudreaux, a policy researcher at Rand, told WIRED he fears schools have rushed to use these tools without considering their limits, expecting the lawsuit to spark calls for instructional environments free of AI and surveillance, grounded instead in human empathy.
