What Norwegians are learning as they pioneer autonomous ships
Norway is leading the world in developing autonomous ship technology – and the world should pay attention to its latest developments
By
Pat Brans,Pat Brans Associates/Grenoble Ecole de Management
Published: 10 Nov 2021
Norway is the perfect place to develop autonomous ships. Norwegians love boats, they love technology, and they love to cooperate. On top of that, autonomous ships have practical applications that could affect the lives of many in Norway.
Mary Ann Lundteigen manages SFI AutoShip, an eight-year programme funded by the Research Council on Autonomous Ships and Operations and 22 partners. She is a professor at the department of engineering cybernetics at the Norwegian University of Science and Technology (NTNU), where the centre is hosted.
“Even though some people think fully autonomous ships exist, as far as I know, all first commercial ships with autonomy will start at degree 2,” says Lundteigen. “Degree 2 means the ship is remotely controlled but has at least one seafarer on board.
“Reaching degree 3 – remote control and with no crew on board – is a bigger challenge, so a test period at degree 2 is a good way to gain experience. And then there’s degree 4, which is fully autonomous – where the ship can operate fully on its own and with no seafarers on board. Degree 4 is out of the question for now – at least commercially. But it is an area of active research here in Norway.”
More research is focused on developing small autonomous vessels that operate in restricted areas. This simplifies the work – and, it turns out, it may provide a solution to a problem many Norwegians face every day.
“A part of our daily lives in Norway is crossing fjords to get to work,” says Frode Halverson, cluster manager for Ocean Autonomy Cluster. “Bridges and tunnels are expensive. Ferries are a better option in many situations.”
Operating several small ferries would be less expensive and more environmentally friendly than operating one big ferry. With smaller ferries, though, the cost of the crew is proportionally higher than for big ferries, so reducing the crew size has a bigger payback. Autonomous ship technology is one way of making ferries smaller and smarter.
A lab to design shore control centres
NTNU is currently working with partners to test a prototype of an autonomous passenger ferry, and a control room to intervene remotely as needed. Unlike self-driving cars, autonomous boats can be run by a remote operator in a cost-effective manner. But when the person controlling the vessel is not on board, a new set of challenges arise, including the fact that the captain may not be the one who goes down with the ship.
“We are making shore-based control rooms to monitor and potentially take over control of autonomous ships,” says Ole Andreas Alsos, head of NTNU Shore Control Lab, which is part of SFI AutoShip. “Our lab does not serve as a control room for a specific purpose. Instead, it’s a shore control lab where we do research on different control room designs. We want to learn how to build the best control rooms for different applications: urban ferries, maritime autonomous surface ships, big deep-sea shipping, short sea shipping, car ferries, and so on.”
The physical design consists of a lot of screens and monitors and a very powerful computer using one of the best graphics cards built for gaming applications – the RTX 3090. Currently, operators inside the control room only get visual and audio information from the ferry. But NTNU is looking at ways of replicating the feel of the ship inside the lab. In the future, this might include haptic feedback, so operators can feel the wind and waves, and if a docking is hard, they will feel it.
Studying operators’ reactions
“Our control room has extra features that help us study how operators behave,” says Alsos. “We have cameras, of course, to see what is happening in the control room. But operators also wear wrist bands so we can measure their heart rate variability and their skin conductance, which indicates their stress levels.
“We use glasses with eye-tracking so we can see where they are looking. That is a good indication of where they have their attention. We also measure their pupil dilation. From the size of their pupils, we can get a good impression of their cognitive loads – a large pupil means high cognitive load; a small pupil means lower cognitive load,” he explains.
Video feeds and sensor data from the control room and operator are sent to another lab, where researchers can speak through a microphone to give operators instructions on what to do. The researchers can also observe control room operators’ behaviours, communication styles and stress levels.
“Autonomous shipping is very new, so we are paving the way. To do this, we collaborate closely with the coastal authorities, who know that ships will gradually become more and more autonomous – not tomorrow or next year, but at some point in the future”
Ole Andreas Alsos, NTNU Shore Control Lab
There is also a large meeting room where video feeds from the experiments or usability tests are displayed. This allows developers, product managers, project leaders and other stakeholders to follow along. This feedback system helps stakeholders assess different control room designs.
“To explore different situations, we built a ferry simulator, which is like a digital twin,” says Alsos. “It behaves exactly like a real ferry, but it has at least one advantage – we can create situations that rarely happen in real life or are too dangerous to test. We can simulate kayakers coming towards the vessel, or people falling into the water. We can simulate fires on board and see how operators react and how much time it takes them to take over the control of the ferry.”
One area of concern is cognitive underload. During long, monotonous trips, the captain on board a ship tends to get bored and may no longer be able to react quickly when needed. The same effect will be even more significant for control room operators, who won’t even be on the vessel. To address this problem, NTNU is exploring ways to keep operators thinking just the right amount. If they do too little, they get bored; if they do too much, they get stressed.
“We might have one person operating one ship,” says Alsos. “If the situation is very complex, we could have several people operating one ship. At the other end of this scale, we could have one person operate several ships, or a team of people operating a whole fleet of ships. The scenario we are currently testing is having two people operate up to 20 small passenger ferries. We are trying to find the best user interface for this specific case.
“There are some international standards on traditional control room design,” he adds. “But autonomous shipping is very new, so we are the ones paving the way. To do this, we collaborate closely with the coastal authorities, who are very proactive and flexible. They are prepared to change some of the regulations to make this happen. They know that ships will gradually become more and more autonomous – not tomorrow or next year, but at some point in the future.”
Artificial intelligence versus rules-based decision engines
“One obvious difference between self-driving cars and autonomous ships is that cars travel much faster,” says Lundteigen. “In a dangerous situation, a car has to intervene very quickly. A ship is larger and slower – and there are, of course, fewer of them. A ship needs much more predictive capability to understand what might happen further into the future to be able to prepare for the situation well in advance. Cars can be brought very quickly to a halt, but large ships take a long time to stop.”
A challenge for autonomous ships is to get them to communicate their state and their intention – not only to the operator, but to other vessels to avoid deadlocks and accidents. To make matters worse, sometimes autonomous ships move in a funny way because they don’t behave like humans. This makes it difficult for human pilots and seafarers to trust an autonomous ship.
“This is an example of ‘explainable AI’ or ‘automation transparency’,” says Alsos. “Complex AI systems and robots and autonomous ships need to communicate their state and future intention to us so that we can trust them and make good decisions based on what they communicate. That black AI box needs to be transparent to us.”
Even though trials have been run with ships using artificial intelligence (AI), it’s still not certain that’s how they will operate when they are commercialised.
“It’s very difficult to test deep learning, because you never know what percentage of the cases you’ve covered with your testing. You train the system on the data you have, from the situations you know. But in real life, variations of those situations usually arise”
Ørnulf Jan Rødseth, Sintef Ocean
Ørnulf Jan Rødseth, senior scientist at Sintef Ocean, believes it will be rules-based or directly programmed. “It’s very difficult to test deep learning because you never know what percentage of the cases you’ve covered with your testing. You train the system on the data you have, from the situations you know. But in real life, variations of those situations usually arise,” he says.
“What I expect is that you will use more rule-based decision-making for anti-collision, and I think a very important element is that the system understands when a situation is not quite predictable. In that case, the system should ask for remote control,” adds Jan Rødseth.
“The main problem in developing an anti-collision system is in trying to guess what a manned ship is going to do,” he says. “Some people say that autonomous systems behave in ways that are difficult for humans to interpret, but the reverse is also true. Autonomous systems have trouble predicting people, especially in complex situations. If all ships were autonomous and they all cooperated, it would be quite straightforward.”
“The system that is supposed to steer and navigate a vessel through dense fairways, interacting with conventional vessels, needs to act in accordance with the Convention on the International Regulations for Preventing Collisions at Sea, 1972 (COLREGs). Data models are needed for systems to recognise situations and react to surrounding traffic.”
Beyond small ferries
“The autonomous small ferry is not the only use case we are considering for autonomous ship technology,” says Alsos. “We are also working on big car ferries that will be partially autonomous. The ferry will plot a course to the destination and follow the route automatically. The ferry will also regulate speed.
“We are looking at auto docking too, where the ferry docks without human intervention. For the time being, we are aiming for partial autonomy. The crew still needs to be present to monitor the systems and look out for other ships.”
Autonomous crossing is expected to save a lot of fuel. Very few people can run the ferry more efficiently than an autonomous system. The other advantage is safety. The captain doesn’t have to control the handles, which frees up his or her attention. All the captain has to do is monitor the systems and look out for other ships.
Thanks to what Norway is doing now, some time in the future, ships will operate without crews. A day-shift captain will be able to go into work in the morning, operate one or more ships from a control room, and be home in time for dinner. When this becomes a reality, the captain will never again have to go down with the ship – and nor will anybody else.
