Case study: Ocado uses machine learning to help customer services deal with email overload
Ocado's head of data, Dan Nelson, reveals details of how the online grocer is using machine learning to bolster the responsiveness of its customer service teams
Ocado has revealed details of how the deployment of machine learning technology is boosting the responsiveness of its customer services team by helping them avoid email overload.
With no bricks and mortar stores to speak of, Ocado’s customer service team is the first port of call for anyone who has cause to raise a complaint or heap praise on the online grocer’s service, be it over the phone, via email or on social media.
“If the customer doesn’t get the order they want, they will go to a shop or an online competitor, so our differentiator is the quality of the service,” says Dan Nelson, Ocado’s head of data. “This is essentially focused on the performance of our customer service centre.”
Roughly 500,000 people around the UK now rely on Ocado to deliver their weekly shop, and ensuring the team maintains its capacity to respond quickly and appropriately to customer queries as its user base grows is a top priority.
This is particularly important when a sudden and unexpected influx of emails to its customer contact centre can occur, such as Christmas or during freak weather events.
Each message the customer services team receives is sifted through manually to assess the urgency and nature of each query so it can be prioritised accordingly and passed on to the appropriate member of staff.
This process can lead to bottlenecks occurring at peak times, and occupies people who should be focused on resolving customer issues rather than triaging an email inbox.
To prevent that, the data scientists within Ocado’s 1,000-strong technology division set to work on finding a way to automate this task using machine learning, natural language processing and cloud computing.
A natural fit
Ocado has been using the Google Cloud Platform (GCP) for big data analytics projects since early 2014, and initially considered using the internet giant’s Cloud Natural Language application programming interface (API) for this project.
The API is used by developers to carry out sentiment analysis and extract details about specific things from datasets, based on knowledge it has accrued from trawling the internet.
“It is a great API, but it is trained around internet data rather than data specific to our business problems,” says Nelson. “So, while it took us a fair way, it couldn’t solve everything.”
This prompted the company to start using the open source machine learning toolkit TensorFlow, paving the way for it to build bespoke machine learning models of its own.
“The team went away, looked at different theorem that would allow them to solve the natural language problem and essentially replicate, but at a large automated scale, the triage and tagging of customer emails,” he says.
Read more about Ocado Technology
A year since computer science was added to the school curriculum, Ocado discovers kids would rather learn to code than speak another language.
The Ocado machine learning model was “trained” by feeding it with three million genuine customer emails that were hosted in Google’s public cloud.
“It is a really precious dataset for us, and allows us to make our model very accurate to the tone and context of our customer,” says Nelson.
“We didn’t know if it was going to take one virtual machine to plough through three million emails or a million machines, and using Google’s cloud, it took away any worries we had about whether or not we had enough hardware capability to do this.”
The model is now deployed within the contact system, where it is used to assess whether the tone of incoming emails is positive, negative or neutral, and tags them so the customer service team knows how high a priority they are.
“It is trained, and now it is in a learning state where we’re making sure it works at full scale and measuring its success rate,” he says.
The measure of success
There are two metrics Ocado uses to assess its performance. The first is “does the model behave in the same way a human would?”, in terms of how it prioritises and interprets the content of the emails, and if it is helping to speed up resolution times.
“The true test will be the unpredictable British weather this winter, and Christmas, which is a big time for any retailer, but also an emotionally charged one,” says Nelson.
“Again, the priority is to deal with customers as quickly and competently as we can. That will be the true test of how the model deals with peak time demands.”
If it manages to hold its own, Nelson says the company could feasibly use the technology to organise customer communications that come in via social media, or pick up on undetected issues developing in other areas of its business.
The priority is to deal with customers as quickly and competently as we can. That will be the true test of how the model deals with peak time demands
Dan Nelson, Ocado
“If we start to see products getting mentioned in emails, we may have a problem developing somewhere in our warehouse, so we can potentially move into fault avoidance,” he says.
“Social media is another place this could work quite well, but the model would need retraining because the language used is slightly different there.”
“Customers buy things at different velocities on different days of the week and at different times of year, so, to avoid bottlenecks, the system optimises the warehouse per day based on what we think customers are going to buy. So it’s constantly looking to rebalance where all these products should be,” says Nelson.
“I am proud of what we have done with machine learning, because it’s not just clever science that’s been implemented for the sake of it. It was collaboration between the business and the technology team to solve a use case together, and we’ve gone from conception to production in a matter of months.”
