Google contractors are evaluating the performance of its Gemini AI by comparing it with Anthropic’s Claude, focusing on accuracy, truthfulness, and verbosity.

Evaluators take up to 30 minutes per prompt to score the models, noting that Claude exhibits stricter safety protocols, often refusing unsafe prompts, while Gemini has been flagged for safety violations.

Internal documents reveal Claude’s responses sometimes explicitly identify the model and emphasize its adherence to Anthropic’s safety policies.

Despite Anthropic’s terms prohibiting the use of Claude for training competing systems, Google has not confirmed securing permission for these tests.

Anthropic declined to comment. Google DeepMind said comparing models is standard practice and denied using Claude outputs to train Gemini, following contractor concerns about Gemini’s accuracy on sensitive topics like healthcare.