Case study: Improving food safety and traceability

Food scandals

The company’s bid to offer traceability in the global food industry began in 2012, when the European horse meat scandal broke. At the time, Mazerolle says Transparency-One began looking at whether people would want such a service and whether it was feasible to offer granularity right down to the individual ingredient across the global food supply chain.

Mazerolle says: “Food scandals have a double effect: they decrease trust in consumer brands, and there is more pressure on these brands to provide greater transparency.”

There is also more legislation on food safety, and new brands with better labelling are able to increase their market share. As a result, Mazerolle says, the traditional brands are adopting traceability in their supply chains.

“We realised there was a real market need and that it was feasible from a tech standpoint,” he adds.

The right tech

But he admits: “If you don’t have the right technology you create a lot of expectations among customers: when it doesn’t work, you face customer disappointment.”

Historically, Transparency-One’s developers were skilled in SQL and Microsoft technologies. So when it assessed how to develop a service that would provide traceability of raw materials for food manufacturers, it had two choices: SQL Server or a graph database.

However, Mazerolle says that when Transparency-One tried to develop its traceability software using SQL Server, it hit a performance wall. “It worked well if we had 50,000 to 100,00 products,” he says, but performance fell as the complexity of the relational database grew.

If SQL Server databases uses a lot of joins to link disparate pieces of information, the system becomes strained.

The same is true during food crises, when every company wants the answer to the same question. “Suddenly you have a lot of people who connect to the system and ask ‘what is the impact of horse meat from Romania?’”

Clearly, a system that cannot scale is not much use.

And it is not just a crisis that can put the database under pressure. Mazerolle points out that managers in global food supply chains have to ask a lot of question on a daily basis to manage risk.

With more food safety regulations, Mazerolle says one of the key questions a product line manager will ask of all factories is if they have the required certification. “You may be asked in the morning to check which factories are not certified.”

Why graphs work

The second option for Transparency-One was a graph database, where the relationship between different data points and the data itself is stored in a graph.

Relational databases, by contrast, work by linking relationships across multiple tables.

Mazerolle says the company looked at whether it should build its own graph database or buy one. “From a management standpoint, our team is not the size of Facebook, Twitter or LinkedIn, which have all built their own graph databases.”

He adds that if the company had tried to build its own graph database, it could have missed out on the market opportunity.

Testing scalability

Among the challenges Transparency-One needed to overcome was proving the product would scale.

“We built a food database using crawlers on the web to get product recipes. Right now this test database has 10 million nodes and it can grow 100 million nodes.” Neo4j can scale to a billion nodes, although the company has yet to push this limit.

There was also the option of using open source. Mazerolle says: “Thanks to Apache, there is a lot of code available, which is beneficial for experimenting.” 

The support question

However, Transparency-One was building a commercial product that would be sold, and Mazerolle wanted reassurance on support. He says that some open source products receive a lot of support from the community, which makes them very solid from a commercial perspective. “But where there is decreasing interest in the community, it is risky to rely on open source support.”

Given that a graph database is cutting-edge technology, Mazerolle says good support was high on his selection criteria. “We wanted real support and customer references on scaling, which is why we selected Neo4j.”

Mazerolle says he asked Neo4j consultants to work in the same office as the Transparency-One developers, operating as a fully integrated team. This improved efficiency, and offered the team the best chance of success as project specifications could be shared.

Upskilling

Working alongside Neo4j’s consultants also gave Transparency-One staff the opportunity to learn graph database skills.

Mazerolle says Transparency-One has progressively reduced Neo4j’s team of consultants and hired new team members of its own. “Today we have internal skills,” he adds. This transition and skilling took the company 18 months.

Going forward, the compnay is looking at extending the services it can provide food manufacturers. New services include risk management and versioning to trace when an ingredient from a particular supplier was being used. 

Since it is able to drill right down to individual ingredients, the company can offer detailed insight into supply chain risks, according to Mazerolle. He adds: “We trace the evolution of a recipe since issues you have today may be due to a recipe used five months ago.”

In the food industry, no two supply chains are the same. Mazerolle says: “We can measure currency exposure and use the World Bank’s country risk index to measure your country risk.”