More eyes watching: How video analytics and AI make ships safer

The life of a captain on a container ship is not an easy one.

They’re in charge of an entire vessel – which could be 3.5x the size of a FIFA football field – and the crew that operates it. They have to make sure that operations are in order, that the ship remains on course, and that it gets from point A to Point B with everything – and everyone – in one piece.

Uri Ben-Dor, founder and CEO of Captain’s Eye / Photo credit: Captain’s Eye

“A ship is a very complicated machine, with a lot of moving parts,” says Uri Ben-Dor, founder and CEO of Captain’s Eye, an AI startup focused on maritime safety and management.

Having served in the Israeli Navy for 27 years, he is keenly familiar with the challenges of operating a ship.

“A captain needs to make sure that everything is running right, everything is working properly, everyone is following procedure, that this machine is working well,” Ben-Dor explains. “There’s a lot to think about.”

The perils of the deep

A lot can go wrong on a ship in the middle of the ocean as a captain tries to keep things in order. Fires, flooding, something breaking down, and someone getting hurt – the list goes on.

According to Ben-Dor, a captain’s life is so challenging mainly because crews have gotten smaller and less experienced even though ships have gotten bigger and more sophisticated over the years.

For instance, the Liberty ship, a class of cargo ship built in the US during the Second World War, had a displacement of 14,250 tons and required a crew of about 50. Today, however, the largest container ships weigh around 200,000 tons but have only 13 to 24 crew members.

This change can be attributed to several factors. Among them are technological developments that have enabled automation, allowing ships to operate with smaller crews. Seafaring is also losing its appeal as a career as younger generations prioritize work-life balance and connectivity.

Container ships have only gotten bigger. / Photo credit: MISTER DIN / Shutterstock

Regardless of the reasons, having a big ship with a smaller crew means that when something goes wrong, it needs to be caught as early as possible. Otherwise, the crew may not be equipped to handle the situation once it gets worse.

“You cannot call an ambulance in the middle of the ocean,” Ben-Dor says. “You cannot call the fire department. You need to handle everything by yourself.”

Most ships are equipped with closed-circuit television (CCTV), and captains frequently make rounds and conduct inspections to ensure that nothing is out of place.

However, CCTV operators need to be alert enough to notice when something unusual has happened. Based on studies, however, a human operator’s attention span wanes after 20 minutes.

“[When I was] a captain, I was going back and forth through the ship at least twice a day, looking to see if there was a problem, if something was not in order or was unusual,” Ben-Dor recalls. “If I could be in every part of the vessel 24/7, I could have probably prevented 95% of the accidents. But I couldn’t.”

What can go wrong on a ship? Click on the ‘X’ symbols to learn more. 

Another set of eyes

All these challenges faced by captains can be traced back to one major issue: a lack of visibility on what’s happening on the ship.

This is where technologies such as computer vision and AI-powered video analytics come in handy. They enhance existing solutions such as CCTV, making it easier for captains to monitor activities on a vessel and act before it’s too late.

Leveraging these technologies is Captain’s Eye, which serves as an additional set of eyes to support captains.

“We wanted to make use of existing systems on board ships to ensure easy installation and implementation,” says Ben-Dor. “We felt that AI video analytics was the solution to detect all these problems using the same camera feed.”

Video analytics refers to the use of AI solutions, such as machine learning, to analyze video footage for different purposes, such as identifying patterns or tracking movement.

Photo credit: structuresxx / Shutterstock

Captain’s Eye trained its solution using video footage of common incidents and issues on ships, allowing it to identify potential hazards more effectively than humans can.

Alongside videos of real incidents, it also used AI to generate images of different scenarios, expanding its training data to enhance the solution’s capabilities.

The solution is installed on top of existing video surveillance on vessels, where it can analyze live video feeds and alert operators accordingly.

For example, Captain’s Eye’s system can detect a fire before it even starts, alerting operators when smoke is detected and allowing them to take action quickly. It also alerts operators to incidents that could lead to fire, such as exposed wires or a leaky fuel pipe.

Without the system, a fire starting from an area that isn’t accessed frequently would likely go unnoticed until the alarm is triggered. At that point, the damage would have already been done, and the crew would need to actually fight the fire.

Photo credit: Yuri A / Shutterstock

This also applies to other issues, unusual activities on a ship. When a vessel gets ready to depart, the crew usually gets busy with the necessary preparations. This may present a prime opportunity for stowaways or smugglers to sneak on board and go undetected for days. But with video analytics, movements as such can be easily spotted.

“We were able to help detect a big drug smuggling event near Colombia recently,” recounts Ben-Dor. “Captain’s Eye detected a group of men boarding the vessel and alerted the captain – he told his crew to be alert, and turned the ship back to port. The smugglers realized they’d been caught and ran off.”

Captain’s Eye also helps streamline operational issues by tapping into satellite tech to enable high-quality video calls between crew on a ship and staff in an office on land.

In the event of problems with the machinery, experts on land can help diagnose the issue and guide the crew in making the necessary repairs, saving a lot of time.

“We’re relevant not only to prevent accidents or security issues, but also in all kinds of operational matters,” says Ben-Dor.

Beyond the oceans

Captain’s Eye has clearly made a tangible impact on its customers by combining video analytics and AI with existing surveillance solutions.

“For one of our major clients, we helped cut the number of injuries on board their vessels by half in 2024,” says Ben-Dor. “Additionally, we were able to help the client save a significant amount of money by identifying technical issues early, allowing them to address these issues before they led to failure.”

The company has focused mostly on the maritime sector so far. However, he sees a lot of opportunity for the tech to make a difference on land as well, especially at ports and factories like petrochemical plants that face similar challenges as ships at sea.

“A lot of the baseline problems and concerns are the same, so there’s a lot of overlap and opportunity there,” says Ben-Dor.

He hopes that as AI and video analytics march on, the tech will become smarter and more efficient, good enough to create a safer environment on both land and sea.

“Humans can’t be everywhere, so we need technology to be our ‘eyes,’ Ben-Dor adds.

The IMDA Spark Programme aims to address the key challenges and support the growth of promising Singapore-based infocomm and media startups by providing selected government tools as well as creating a vibrant and collaborative ecosystem and network.

Is your startup looking to tap into the opportunities the city-state holds? Learn more about what the Spark Programme can do for you on its website.

JOIN IMDA SPARK

Captain’s Eye is a cutting-edge real-time video analytics solution powered by an advanced AI algorithm that can be applied across vessels, warehouses, ports, factories, and other facilities. Learn more about Captain’s Eye on its website.

This content was produced by Tech in Asia Studios, which connects brands with Asia’s tech community. Learn more about partnering with Tech in Asia Studios.