Unilever provides recipe support to blind and partially sighted shoppers

Unilever has worked with accessibility app developer Be My Eyes to create tools for shoppers who are blind or have low vision.

The initial trial uses the Be My Eyes technology in the UK on Colman’s Singapore Noodles Meal Maker, with the intention of rolling it out across more brands in 2024. Scanning the accessible QR (AQR) on the front of the pack, developed by computer vision specialist Zappar, enables shoppers to access the Be My Eyes app, which either connects to a volunteer to read and describe cooking instructions or a virtual artificial intelligence (AI) chat bot powered by ChatGPT-4 to address recipe or cooking questions.

Be My Eyes said this is the first time its technology has been integrated with a food product, offering an AI-assisted cooking experience at home. Unilever is working with Be My Eyes to train its volunteers, and provide specific Colman’s product information and data to the ChatGPT-4 AI bot. Unilever’s customer support is also integrated into the Be My Eyes app experience. This enables shoppers to connect automatically to the Colman’s careline team for further support if needed.

Rachana Dongre, senior digital engagement and strategy lead for nutrition and ice cream at Unilever, said: “We’ve accelerated digitising our packs to offer new opportunities for brand engagement and elevated shopping experience, and now we’re also focusing on how we can use digital experiences to make our products more accessible.

“Zappar’s AQR codes mean we can support blind and low-vision shoppers to have equal access to information, and integrating Be My Eyes into these codes offers a totally new way to make the full experience of our products more inclusive, from the shopping aisle right through to cooking at home,” she added.

Discussing the challenges facing people who are blind or have low vision, Be My Eyes CEO Mike Buckley said: “The difficulty in reading package ingredients and cooking instructions has long marred the cooking experience of these consumers. Unilever’s commitment to using new Zappar technology with Be My Eyes is pioneering a new era in product accessibility and disability convenience.”

Caspar Thykier, Zappar’s co-founder and CEO, described the technology as “a new type of product as a service”.

Unilever began adding AQR codes to its products last year, including Colman’s and Persil products in the UK. The AQR codes, which are developed by Zappar, work by adding a dot-dash pattern around one corner of the code, which makes them detectable from further away.

The Zappar AQR code can be scanned through accessibility apps from 1.1 metres (over seven times that of normal QR scanning) to help blind consumers find their desired product in store or at home.

Unilever uses these AQR codes to provide product information such as usage direction, recycling, and ingredient information and nutritional value, interacting with standard smartphone accessibility through the Be My Eyes app to share information via audio description or displaying information in larger text.

Unilever plans to work with Be My Eyes and Zappar to collate user feedback to optimise the technology with the aim of rolling out the accessible technology to other Unilever products. It has also said it would like to continue to partner with accessibility experts and apps to support wider integration of AQR technology across different products, categories and countries, and work with other businesses using this technology to share best practice.