Interview: How Defra is getting its IT systems ready for Brexit

The EU Exit projects

The idea from the outset was to limit the amount of new systems, as the department knew it had taken on “a huge task”, according to Broomfield.

“Wherever we could, if we identified that an existing system could be tuned and hardened to cope with increased transaction volumes around trade, we would do so” he says.

“For example, in Plant Health Portal, that was the route we chose, with safe and lower-risk approaches to test and determine how the system would perform under the expected loads and if we needed to harden it,” he adds.

More data-related services were introduced under the programme, including a new warehouse and business intelligence (BI) applications set up to report data from some Brexit services.

“A reference data service, which aligned with Defra’s strategy, has also been introduced and is about getting our reference data held in one place and maintained effectively,” says Broomfield.

Out of all the new systems based on Microsoft cloud capability introduced as a result of Brexit, there are six core platforms: Imports of products, animals, food and feed system (IPAFFS); export health certificates; registration, evaluation, authorisation and restriction of chemicals (Reach) IT; catch certificates; UK F gas system; and veterinary medicines digital service (VMDS).

These systems focus on trade, specifically relating to the UK borders, and are among the most crucial for the department.

“We’re currently using EU systems for some of those border functions and we had to assume we would no longer have access to those systems,” says Broomfield.

“We built a new system for the controlled imports of products that Defra has an interest in, including live animals, products of animal origin, food and feed,” he says, adding that this required building a new system, which was ready to be used on 29 March 2019 if required.

On the other end of the telescope, there are systems relating to exports. According to the IT executive, Defra had to assume that the UK would be treated as a “third country” by the European Union after Brexit.

“This could mean if you’re not in the EU and you are exporting into the EU, you have to produce export health certificates on a number of products such as animals and food,” he says.

This also means a very rapid increase in the number of export health certificates that Defra would have to authorise and support production of, so the team also had to build an export health certificate system.

Registration is another area of focus for the new core systems, covering chemicals and veterinary medicines. At present, Helsinki-based European Chemicals Agency handles chemicals registration, while the European Medicines Agency deals with the registration of veterinary medicine.

This meant the IT team needed to understand what would be required under a UK-based regime to register and control those products and the two new IT digital services required to support them.

In addition, there is work related to the control in the trading of quotas around fluorinated gases and ozone-depleting substances, which is also currently managed through interactions with the EU.

The last major core IT system Defra had to build supports fishing exports. To export to Europe, non-EU suppliers have to be compliant with a number of regulations around sustainability, so a new digital service needed to be built to issue official catch certificates, which enables fishing businesses to provide authorities with information such as where the fish has been caught and if it has been caught within quotas

“We tried to identify if we could manage with an interim service that was lower-risk to deliver, then do that first – we did that for our export health certificates, for example,” says Broomfield.

Readiness and risk

Concerns have often been raised in the past couple of years over Defra’s ability to deliver the IT required for Brexit in time. Broomfield does not try to play that down, adding that there were good reasons to be anxious.

“Whenever you’ve got a time-critical project such as EU Exit and you’ve got new IT services being built, of course you’re right to be concerned. Those of us who work in this space know that major IT deliveries have a lot of variables and risks that can manifest themselves,” he says.

“So it was right that people were worried about it. Colleagues in Defra were also concerned.

“One of the things we did to address that risk was – and our ministers were very clear on this as well – having contingency measures in place if those IT services weren’t going to be ready. That was a very positive measure to de-risk day-one readiness.”

In practice, having contingency in place means that if a system such as the one handling imports not been ready, imports from third countries such as the UK would no longer be able to use the EU system.

That required Defra to think about what contingency would look like and how import agents would tell the department about animal origin products coming into a particular port.

The exercise involved looking at what that manual or semi-manual process would be, generating forms that would be available on Gov.uk and setting up email addresses to major points of contacts, so that importers could email the forms in.

For agricultural controls on the Irish border, the same principles apply in terms of how Defra would deal with EU imports coming into the UK on day one, as any inspections required would take place away from the border.

According to Broomfield, the overall volume of transactions is going to be to some extent based on modelling because not all of these transactions are recorded at the moment.

“If export health certificates are taken as an example, we predicted it would be between 100% and 300% increase, and we obviously tested to the 300% worst-case level,” the says.

“On the import side, that was not as significant because we already understand the number of notifications from third countries outside the EU.”

The cloud advantage

The potentially substantial increase in transactions post Brexit means that building systems for the cloud was an obvious choice.

“You have to architect systems in a way that will allow you to benefit from that, which is why we had an architecture program and worked very closely with Microsoft to ensure we were built for elasticity,” Broomfield said.

In addition, an early realisation when planning for Brexit was that all projects would have common requirements such as identity management, customer record management, integration services and a cloud computing service centre.

Defra then took advantage of its investment under its ongoing digital transformation strategy UnITy around Microsoft Office 365, SharePoint and cloud, which has enabled the department to quickly build platforms that would suffice for day one of Brexit, and in parallel, build sustainable, new services for the long-run. 

“What I did was accelerate the development of those common services. Looking back, it’s the right thing to do in terms of where we’re trying to get to in Defra,” says Broomfield.

“But, of course, it did introduce a dependency for multiple projects that are all reliant on a new service coming in – if that service is disrupted or delayed, it affects other projects.

“Our philosophy was to get automation of our testing and our quality assessments embedded as early as we possibly could”

Jo Broomfield, Defra

 
“So we had to put a lot of attention into driving forward delivery of those common platforms to make sure that they were ready and tested in time to integrate them with new services.”

Based on some of his past experiences, things that Broomfield recognised as potential sources of trouble in such a project are around the space of cloud management, environment management and deployment management. He sought to prevent such issues with a digital delivery pipeline.

“We built the infrastructure, setting up those cloud environments that we were going to have in our pipeline, and then looked to automate as much as we could. We built for automated deployment and then built in the quality assurance and testing to get as much automation in place from the start,” he says.

“As we brought our partner supplies on, we had a set of templates or patterns we asked them to use, so we could get assurance that we weren’t going to get caught at the end of the process, which often happens with testing.

“Our philosophy was to get automation of our testing and our quality assessments embedded as early as we possibly could. That evolved over time, but it’s one of the key things that’s enabled us to deliver, prioritising [the creation of that framework] as one of our first tasks.”

Present and future work

When Defra started the Brexit IT programme, it had to manage its ambitions and accept that it wasn’t going to be able to build capability beyond the bare minimum needed to operate once the UK left the EU.

The department focused on building minimum viable products (MVPs) under an agile approach.

“It was very much about keeping our scope controlled onto those minimum viable products for day one,” says Broomfield.

“But we had things in our development backlogs beyond the MVPs. Since the end of March, our focus has been to continue to tune and harden the systems – it’s just that the deadlines have moved again.”

Even if the UK was not exiting the EU, what Defra is doing now would have continued anyway, says Broomfield, because it improves user experience and adds functionality.

“Improving the service wrapper, some of the data analytics and reporting, for example, were things we knew we could improve afterwards,” he says.

As for the future of the function, despite the current ambiguity around EU exit plans and scenarios, Defra’s IT leadership is clear that it has to be ready again for 31 October, if necessary.

“There will be a lot of focus into [getting ready for the next deadline], with some of the projects we knew were going to take longer than that initial period to build anyway,” says Broomfield.

Projects that are currently ongoing and benefited from the prolonged delays around Brexit include the chemical system, where the day-one requirement for was to allow registration applications and registrations in.

However, back-office functions that would be processing people’s applications and specific scientific review wasn’t part of that initial requirement, but something that would have to happen in the period following the exit.

Another challenge the department faces is that some of the trade systems it decided to use and harden for day one are quite old.

“We’re looking at all the options for bringing [the functionality of] those services into the new trade system,” says Broomfield.

“We will do that so we can get the benefit of modernisation and transformation of those services, some of which will lead to more efficient processing, in areas such as endangered species coming through the border.

“This work doesn’t come to a cliff-edge stop. We expect activity to carry on at a lower level through to the end of this financial year. And then as part of the planning process that Defra will undertake, we’ll be looking at whether there’s more activity that needs to go on beyond that.”

Structuring resources

At its peak, the EU Exit delivery team team had just over 500 people, including civil servants, contractors and supplier staff, and is currently down to around 300 people.

The team includes WebOps staff, people supporting the cloud service centre, as well as staff focused on security, data architecture, data warehousing and artificial intelligence (AI).

According to Broomfield, the set-up isn’t in place just to deliver the IT systems needed for Brexit.

“It’s also about the service wrapper and all the other things we wanted to do around it, such as integrating our service management with the new arrangements being put in place with the UnITy contract,” he says.

Defra had to find a location for all the extra hundreds of new staff. Alongside CDIO John Seglias, Broomfield scanned the estate to find an office.

“John and I initially thought we were going to need at least a couple of hundred people, and the only free space we could identify was an office in Crewe, which we then refurbished to make it suitable for an agile team,” he says.

Despite concerns over whether Crewe was the ideal location to set up a new high-tech IT operation and if it was going to hinder recruitment, the department found staff could get there easily from Carlisle, where one the key Defra locations is based, as well as London and Bristol.

In addition, staff were recruited from Liverpool and Manchester – areas Defra traditionally did not pull people from.

However, Broomfield and Seglias soon discovered that the 200-space office accommodation wasn’t big enough, so the next issue was how to expand.

“Very quickly, we had to expand the estate across the UK to support the work we were doing,” Broomfield said.

The department then sought to co-locate some delivery teams with its internal customers, with catch certificates teams working at the Marine Management Organisation in Newcastle, and animal medicine teams co-located in Weybridge. A Defra location in Bristol was used for the common platforms.

Suppliers’ offices were also used as Defra could not find enough spaces for the people working on IT supporting Brexit. An office in Birmingham was set up with Kainos, while Capgemini offered an office location near Manchester.

The partners we have engaged have absolutely committed to the same goals that Defra has in terms of readiness

When it comes to suppliers, Broomfield says that adopting a novel approach to resourcing, as well as ensuring the department was catered for in terms of its many requirements, was key.

“We anticipated that our IT partners and suppliers would be getting quite a lot of demand from other parts of government as well, so it was important to have a flexible approach to how we resourced capability in,” he says.

“The partners we have engaged have absolutely committed to the same goals that Defra has in terms of readiness. Conversations took place at CEO level with some of our key suppliers to make sure that was the case.”

Having a mix of suppliers was also helpful, according to Broomfield. This included incumbent suppliers as well as going to market through Cabinet Office frameworks for niche specialist areas, from smaller suppliers such as KITS and Automation Logic.

Leadership lessons

Lessons learned at a leadership level from work carried out under the EU Exit unit so far are around how to suddenly put together a capability of that size in a short period.

“Creating an environment of collaboration and not allowing defensive posturing between different players was key,” says Broomfield.

He adds that bringing some supplier leadership into Defra’s senior teams was also crucial to ensure delivery against timescales – this became even more evident as the deadline approached.

“When we got into the intense six months before the go-live date, having the suppliers on those operation centre daily calls, we learned to challenge each other constructively to get risks on the table, but also to collaborate really well,” the says.

While Defra was keen to get a clear view of the risks involved, it was also conscious of the impact the intense schedule could have on teams’ mental and physical health.

“People’s wellbeing was a very important point for Defra in that process, so knowing that they were going through a really difficult period, that was always on the agenda,” says Broomfield.

“We were always asking ourselves the question, ‘Can we give people a break this weekend?’. There is a tension there because if you don’t work that weekend, it’s lost because you don’t get the time back.

“It was about trying to judge to what extent we needed the foot to the floor. In any case, we managed through that process because we just had tremendous commitment from people.”