Cute AI agents need clear boundaries

Welcome to The Prompt, your guide to the biggest issues and developments shaping AI, with Asia front and center. Want full access to all our AI reporting? Subscribe here.

Hello reader,

Even Mark Zuckerberg, an insanely competitive person, wants his AI agent to teach him how to win an MMA fight. On a recent podcast, the CEO of Meta revealed that he uses Muse – the company’s AI agent – to analyze footage from cameras in his gym and offer advice between rounds.

His biggest complaint? Muse takes 30 seconds to respond, eating up half his break.

Alexandr Wang, Meta’s chief AI officer, has a different priority with Muse: getting it to source obscure teas from Taiwan and Japan.

Meanwhile, former WSJ journalist Joanna Stern used Muse to book a massage appointment. She has been using the agent to perform what she calls “life-admin jobs” such as track flights and cancel subscriptions.

Meta’s slouchy, cutesy AI agent hasn’t reached Southeast Asia, so I haven’t tested it myself. But as fears grow over AI agents becoming harder to control, there’s something ironic about tech companies packaging them as adorable little companions.

According to Meta’s design team, a customizable avatar feels more natural to chat with than a corporate logo. And it isn’t the only player giving AI agents a cute makeover.

OpenAI’s Dots lets users customize their agent’s eyes, glasses, and accessories. Meanwhile, Town’s AI assistants include a bespectacled bear and a lobster.

Image credit: Made by Ulla/Tech in Asia with the help of AI

But do users realize how much authority they’re handing over to these cute little creatures? In an essay, digital safety analyst Allison McSorley argues that Muse’s cute appearance makes users less wary of sharing personal information. She calls it a “cuddly Trojan horse.”

And there’s more to Muse than its adorable exterior. Independent testing suggests that its underlying model, Muse Wonder, shows improvements on complex tasks such as writing code and turning large collections of source files into spreadsheets and presentations.

That said, being good at complex tasks doesn’t mean an agent will respect permission boundaries. Take prompt injection: malicious instructions hidden in an email or webpage could trick an agent into forwarding private information.

Meta says its safeguards include isolating the agent, keeping credentials out of its reach, and using a separate system called Sentinel to approve or block actions.