Baidu’s image-recognition software isn’t perfect, but it’s super smart and will make you laugh

Baidu, the Chinese internet giant that made its name with search, just released an updated version of its Baidu Translate app yesterday for iOS. In addition to the usual type-input, voice-to-text translation, and Optimized Character Recognition (it will translate your friend’s T-shirt for you), the update comes equipped with image-recognition software.

That means you can go outside, take a photo of a fire hydrant, and Baidu Translate will display the words “Fire Hydrant” along with the Chinese “消防栓”.

Baidu tells us that the souped-up Baidu Translate marks the broad introduction of the company’s image-recognition technology, though in the past it has dabbled in image search for narrow fields like dog breeds and album covers. The bulk of the research behind the app took place at the company’s Institute of Deep Learning in Silicon Valley, where top data scientists have been working to integrate nimbleness and complexity of the human brain to into confines of computer algorithms.

Curious to see how these new features measure up, I took the app for a test drive in one of Taipei’s best knick-knack filled cafes.

For the image-recognition feature, I started out simple – a coffee mug. Here’s what Baidu came up with.

Okay, not bad – we’re looking at a coffee mug, so we’re in the ballpark. But that first test highlights one of the weaknesses of Baidu translate, and a rather unexpected one at that – it shoots beyond the mark. Rather than coming up with “cup” or “glass,” Baidu churned out a number of different frou-frou beverages. What if I was drinking tea? “Shannon Coffee,” meanwhile, appears to be a literal translation of the Chinese, which out to be interpreted as “Rich, fragrant coffee,” but that’s an issue for the translation team, not the image team.

Here’s another example of an overshoot:

And another:

Can it help me read the menu?

Baidu Translate: ‘Caption This’ edition