Dutch national weather service adopts cloud to expand service offerings

The Royal Netherlands Meteorological Institute (KNMI) is driving a cloud-based infrastructure refresh as the foundation of a modernisation programme aimed at creating opportunities for weather data applications.

In addition to being a weather forecasting and monitoring service, the KNMI is a national research and information centre for weather, climate, air quality and seismic activity. The agency is now planning an early warning centre to provide real-time data to stakeholders from air traffic controllers to private sector businesses.

Climate change and an increasing need for weather-related information meant that the agency had to up its game in terms of infrastructure, according to the head of information and process management at KNMI, Jan-Willem Arnold.

“There is a huge demand for weather data. We can do weather forecasts, but we are much more interested in the impact of the weather, and it is possible to collaborate with all sorts of organisations based on that information,” Arnold told Computer Weekly at the Amazon Web Services (AWS) Public Sector Summit in Brussels on 9 April 2019.

“However, having near real-time weather information means very intensive calculations and a lot of computing power to be able to meet those requirements,” he added.

KNMI’s measuring network, as well as its weather and climate models are optimised and refined in an ongoing basis and run on a supercomputer introduced two years ago. Such calculations form the basis of the weather forecasts, climate scenarios and products offered by the agency.

With the exception of production loads run on the supercomputing facility, all the agency’s systems with high availability requirements are being shifted to the AWS cloud. According to Arnold, the migration should conclude by year end.

“We are moving [to AWS] because of the agility that the cloud provides. Demand for specialised staff [to support privately hosted systems] is huge and you can’t get these professionals in Holland, particularly around the Amsterdam area,” Arnold said.

According to the IT chief, the AWS set-up is significantly more cost-effective than using in-house resources. This proved to be the case from the outset, when KNMI used the supplier for EU-funded research projects that required more agility.

“With AWS, I could take a pay-as-you-go server and cancel any time. If I was to use the government shared services centre, it would take about a month or two to set it up and pay a lot more,” said Arnold.

However, there are challenges around the migration. According to Arnold, cloud computing is relatively incipient in the Netherlands, and decision makers often express concerns about its ability to deliver, its security and loss of control of IT assets.

“It took between six to nine months [to convince stakeholders to adopt the cloud approach] between policy writing, talking to people and assuring everyone that it was OK to make that move from a digital government perspective,” he said.

Arnold observed that the fact KNMI works based on open data about the weather rather than individual information on citizens was a helpful point in taking these conversations forward. He added that the cloud introduced a real organisational change in how things are done, particularly when it comes to rethinking the cost of products.

“Since we adopted AWS, we are tagging all the resources we use to a specific product. So if you are the owner of the project, you will get the bill, whereas previously I got the bill,” he added.

While it is too early to estimate the savings from the migration to the cloud, Arnold expects the benefits are more related to additional business agility.

“I expect cost savings, at least in some aspects, but I now need a cloud operations team, so I will have to train people and Amazon specialists have to be hired externally – and they are not cheap,” Arnold said.

Within the next 12 months, Arnold expects his datacentre basement to be empty, with the cloud set-up providing “much better business continuity guarantees”. He also hopes the migration will have changed the IT organisation and facilitate a shift to DevOps.

In addition, the executive expects the early warning centre will have been designed, providing an open data digital government platform that will enable stakeholders across Europe to do more with real-time weather data.

“But tidying up the house is the first step. We are now busy with the groundwork, which is about educating people in the institute about these new ways of working, so they can look at this new horizon and wonder what else can be done with the cloud.”