Big Tech warned over chatbot harms by US attorneys general

Forty-two US attorneys-general have called on major AI companies to strengthen safeguards and testing for chatbots, citing concerns about user harm.

Recipients of the letter include Google, Meta, Microsoft, OpenAI, Anthropic, xAI, Perplexity, Character.ai, and Replika.

The attorneys-general said some users have formed damaging emotional connections with AI chatbots, leading to alleged harms including involvement in wrongful death cases.

The letter highlighted at least six deaths, including two suicides and a murder-suicide, allegedly linked to chatbot interactions.

Officials warned that failing to add protections for children and address misleading or sycophantic outputs could violate state laws.

The intervention comes as President Donald Trump has said he will issue an executive order this week to limit state-level AI regulation, amid concerns from tech firms about navigating multiple state rules.

The attorneys-general urged companies to improve safety testing and separate revenue goals from model safety decisions.

🔗 Source: Financial Times

🧠 Food for thought

Implications, context, and why it matters.

Executive orders face limits on overriding state enforcement authority

Trump’s planned executive order to curb state AI rules runs into basic law. Federal preemption means federal law overrides state law and it usually needs Congress. State attorneys general (AGs) get enforcement power from state constitutions and legislatures, which an order cannot cut back.
In a letter, 42 attorneys general (AGs) say chatbot harms fit consumer protection and child safety laws. A federal order could set standards that make state cases harder, but it cannot erase statutes or stop AGs from filing under current law. AI makers should expect active enforcement in Utah and New York with chatbot laws.

Safety certification market grows as AI firms seek compliance cover

AGs want safety testing independent of revenue targets, which drives demand for third-party audits and certifications to prove compliance. International Organization for Standardization (ISO) 42001, the world’s first AI management system standard, offers a framework on ethics, transparency, and ongoing validation 1.
Accredited bodies such as Schellman and DNV now audit against ISO 42001, which creates openings for AI governance tools, safety testing platforms, plus compliance documentation systems 1 2. Training firms run Lead Auditor programs, credentials for professionals qualified to lead compliance audits 3.
AI infrastructure vendors and security firms can build ISO 42001 aligned audit services or partner with accredited bodies. That gives a near term go to market path while buyers seek defensible safety processes before possible state actions.