There are few things in the retail industry that will generate more negative sentiment and attention from the media and privacy campaigners than the use of facial-recognition software in stores.
There are major opportunities to be potentially derived from artificial intelligence (AI)-powered facial recognition software, but could the retail industry miss out on many of these because of the technology’s negative and controversial reputation?
Fraser Sampson, biometrics and surveillance camera commissioner, recently delivered a major paper on dealing with this delicate balance to the home secretary, stating: “We are on the precipice of a technological revolution in the way we do things.”
This requires action, he argues: “We urgently need to wake up to the opportunities presented, and the threats posed, by the explosion of capability in AI-driven biometric surveillance. If we fail, we risk missing out on the potential benefits it can offer and exposing ourselves to the potential dangers it poses.”
He acknowledges that live facial-recognition (LFR) technology can be intrusive to privacy, but he also believes it can combat serious crime and abuse. Sampson has concerns that the fear of “blowback” from the media and other groups is a major deterrent to businesses considering investing in this area. This is not helped by the challengers to the technology often using old data that was generated when the technology was less advanced.
Simon Gordon, founder and chairman of Facewatch, is well aware of the challenges, having launched his facial recognition solution in 2018 when the General Data Protection Regulation (GDPR) became law in the UK. Facewatch is currently the only GDPR-compliant facial-recognition service in the country and is deployed by a wide variety of retailers including Budgens, Costcutter, Frasers Group, Eat 17, and Southern Co-op.
The solution involves cameras pointed towards the retailers’ doors and their video feedback processed onsite by Jetson Nano edge box devices. When a face is detected, it is sent to the cloud for biometric processing using AI technology and two algorithms that compares the image against a facial database (containing images of “subjects of interest” who might have stolen from that store before or from another store within a selected radius).
The facial image of a customer not on the database will be deleted immediately. If there is a match, then an alert is sent directly to the store. A final check involves the store staff verifying this match, which overall gives an accuracy of 99%. This might edge up further in the future as Facewatch is investigating cameras with the AI built into them, thereby removing the need for processing by an edge box in-store and, according to Gordon, “facial recognition will combine with gait analysis and other methods to improve accuracy”.
It always seemed that [shoplifters] were a step ahead of us, but now with Facewatch in the store, I feel I have the cards in my hand to fight them
Emanuele Jardin, Eat 17
Gordon talks of the upsides for retailers from such a solution, which he says “protects data and manages it proportionately and fairly”, adding: “In reality, the only people who make a [negative] noise are the lobby groups. We’ve not had a single complaint from customers.”
There is certainly no complaint from Emanuele Jardin, store manager at Eat 17 in London’s Hackney, who says: “We can see a big difference – no big shoplifters any more. They think, ‘We can’t go there anymore because it’s not worth it’, because they’ll get caught every time. It always seemed that they were a step ahead of us, but now with Facewatch in the store, I feel I have the cards in my hand to fight them.”
For most retailers, Gordon says the solution is about preventing a crime, which then stops any potential conflict in-store. “The primary role of the technology is to make the store safe and it saves money [from reduced theft levels]. It makes the store safer for customers and the staff feel safer too,” he says.
Creating a safe environment is the primary reason Southern Co-op implemented the technology and, according to Gordon, they have defended using it in selected stores that have the worst reputations with higher levels of theft and abusive behaviour to its shop-floor employees.
“[The Co-op] have gone through the barrage and once it’s over then there will be no more bad publicity,” adds Gordon.
Taking advantage of age-estimation technology
The need to protect employees is fully understood by Graham Wynn, assistant regulator for business and regulation at the British Retail Consortium (BRC), who says: “Our most recent crime survey revealed that instances of violence and abuse are unfortunately all too common for retail staff, and cases have been rising year after year.”
He points out that most conflict occurs when staff challenge customers purchasing age-restricted items: “While occurrences peaked during the pandemic with1,300 a day, cases have been rising each year. Age-estimation technology was found to be an accurate and useful tool, as customers in the trial were much more willing to accept the challenge from a machine, therefore reducing the risk for frustration to be directed at individuals.”
This is where care has to be taken with descriptors because facial age-estimation technology should not be confused with facial-recognition technology. Wynn is keen to clarify the point: “The age-estimation technology does not create privacy issues as an individual’s facial characteristics are not stored by the retailer or the machine, they are only used to assess the age of the customer.”
Julie Dawson, chief policy and regulatory office at Yoti, is also keen to differentiate the two as she does not want the Yoti solution to be thrown into the same bag as facial recognition and its negative reputation.
The company recently played a major role in a Home Office-backed six-month trial involving Morrison’s, Co-op, Tesco and Asda that investigated the potential to include digital technology solutions within forthcoming alcohol licensing legislation, which clearly includes age restrictions.
The retailers installed the Yoti software on selected self-checkouts around the country that involves an AI algorithm used to estimate a customer’s age from facial scans taken by the in-built cameras on the checkouts. Dawson says: “The tech is trained with a huge dataset and involves pixel-level analysis to give an age estimation of the customer. For people aged between 13 and 19, it is accurate to within 1.5 years, which is better than a human who is only accurate within six to eight years.”
Age-estimation technology was found to be an accurate and useful tool...reducing the risk for frustration to be directed at individuals
British Retail Consortium
With no personal data saved and the solution reducing friction in stores as well as helping to reduce queues, as many as 70% of people have stated they would use facial age estimation in stores if it is an option, according to YouGov.
As well as being used in the Home Office trial, the Yoti software is used in the maiden Aldi Shop & Go store in Greenwich when customers are purchasing alcohol. Dawson also points to the technology being potentially used to verify ages at Click & Collect lockers when customers are collecting age-restricted products, and for online transactions when the camera in PCs and smartphones can be utilised for the facial scans.
Matt Redwood, vice-president of retail technology solutions at Diebold Nixdorf, recognises the value of these age-estimation solutions at self-service checkouts, with as many as 22% of all transactions requiring age verification. As well as using the cameras on the tills to scan faces, he says they are being used overseas by companies for product recognition.
“We have it ‘live’ for fresh produce recognition. Fruit and veg, and bakery, is recognised by an AI camera above the scale and the consumer simply has to confirm the product has been identified correctly,” he says, adding that the next big step in-store is for more process-tracking to stop theft at the checkout. Dedicated cameras above the tills identify movements that suggest intentional miss-scans, scan avoidance and barcode switching.
Redwood says this is a different dynamic to facial recognition as people do not feel like it is actually watching them personally. The same thing appears to be happening with the self-service checkouts that show a screen that plays back in real-time video footage of the customer as they use the till. These are currently being used across the UK in the major supermarkets.
“It’s clearly used to stop people from stealing, but it’s not being analysed as such and no customer is being identified with it so people are okay with it. It’s just like CCTV. But if the retailers tried to add facial-recognition software onto this, then it would be a nightmare for them,” he suggests.
He believes it might be an easier proposition to introduce facial recognition for staff to log on at work, but even here he predicts there would be arguments around the issue of the companies holding information and pictures of their employees and how exactly this is to be used.
“Facial recognition is a long way off being used in retail. From a technology perspective we could do it today, but retailers have to work within the adoptive curve of consumers and at the moment they are a long way off [adopting] it. Product recognition, age verification and process tracking are more obvious early uses of these types of technologies, believes Redwood.
This is not necessarily the experience of Facewatch, of course, and certainly the commissioner Sampson will be hoping that the widespread adoption of such solutions is not quite so far off, provided the pros and cons of the technologies involved have been fully considered by all stakeholders, allowing consumers and retailers to enjoy the upsides.
Technology is becoming an important part of the shopping experience at home, in stores and in-between, so how are retailers using technology to advance the store experience?
