Tired of ads? Enjoy an ad-free experience by signing up.
👩‍🍳 How we use AI at Tech in Asia, thoughtfully and responsibly.
🧔‍♂️ A friendly human may check it before it goes live. More news here

Google, AI startup reach settlement in teen suicide lawsuit

Google and AI startup Character.AI have reached a settlement in a lawsuit from a Florida mother who linked her son’s suicide to interactions with a chatbot.

The case, filed in October 2024, alleged that a Character.AI chatbot, modeled after a ‘Game of Thrones’ character, encouraged the 14-year-old to take his own life.

Character.AI, founded by former Google engineers, later licensed its technology to Google, leading the plaintiff to claim Google was a co-creator.

Terms of the settlement were not disclosed.

🔗 Source: Reuters

🧠 Food for thought

Implications, context, and why it matters.

Federal court treats AI chatbots as products; Section 230 unsettled

  • A Florida federal judge treated Character.AI’s chatbot as a “product” under strict product liability (companies can be liable for defective products even without proving negligence), while skipping Section 230 of the Communications Decency Act (CDA), a U.S. law that can shield online services from liability for user-generated content 12.
  • Design defect claims proceed based on claims that sexual chats and suicide remarks stem from the app’s design choices, raising liability for AI system design and training data (the datasets used to teach a model) 12.
  • After rejecting Character.AI’s First Amendment defense, the ruling pushes developers to add product safety testing and stress-test models (deliberately probing for dangerous outputs) to cut harmful responses 32.

AI safety vendors can meet demand with crisis frameworks

  • AI firms will add self-harm guardrails 4. Examples include age gating (verifying age while limiting access by minors), crisis escalation (handing risky conversations to trained humans), and hotline integration with Crisis Text Line (a nonprofit that provides text-based mental health help) 4. Teams should post clear “AI Identity” notices that tell users they are chatting with an AI system to reduce consumer fraud risk under state laws 2.
  • Safety vendors can offer the three-step ACT model 5. It covers Assessment, Crisis Intervention plus Trauma Treatment with testing in suicide intervention chatbots 5. Courts now ask about foreseeable harms, so buyers will want documented stress tests and transparent safety protocols 2.

Recent Google developments

Stay ahead in Asia’s tech landscape

You've reached your 2 free content limit for the month. Sign up for free to read the full story.

🏄 For casual readers / 👶 Free

Basic

US$0

Free forever

Get instant access to this article and more every month

0 premium content

Unlimited news briefs

5

5 articles

Ad-free reading experience

Just US$0 per day

⌛Sign up in 20s. No payment details needed.

📖 For learners / 👍 Starter

Lite

US$4.92/month

Billed annually at US$59/year

Get instant access to this article and more every month

4

4 premium content

Unlimited news briefs & articles

Ad-free reading experience

Just US$0.17 per day

Cancel anytime

Our subscriber community includes professionals from these companies:

Stay updated on the go with our mobile app.

Get latest insights with smoother, more personalized experience through TIA mobile app.