Retailers mired by manual data integration

Almost half of the 252 senior data decision-makers in UK retail and e-commerce who took part in a recent Censuswide survey still manually input third-party data sources into their own systems.

External data sources are an increasingly important part of a retailer’s strategy to remain competitive. For instance, retail and e-commerce businesses implement dynamic pricing strategies. But staying ahead of the competition is increasingly difficult and simply changing prices on products is no longer enough. Data is necessary to understand and implement efficient pricing strategy.

The survey, commissioned by Oxylabs.io, found that 43% of respondents will use data for predicting market trends, and 39% will use it for forecasting consumer demand and benchmarking against competition. Other uses of external data include improving customer service (38%), optimising pricing strategies (37%) and product development (35%).

The survey also reported that the technique of scraping data from websites is too complex to implement. More than one-third (38%) of respondents listed “budget constraints” and 35% cited “lack of talent” as reasons for not implementing web scraping and cleaning for data collection.

The survey also found that 44% of respondents in the retail and e-commerce sector are concerned about the legal complications of web scraping.

According to Oxylabs.io, although attempts have been made to educate organisations about the legal nuances of web scraping, the message still needs to be communicated further across the industry. The study reported that 59% of companies have hired internal compliance teams to mitigate risk, while a smaller number hire external legal and compliance teams. While better than nothing, external teams might not be as efficient, especially at large-scale scraping, said Oxylabs.io.

The survey showed that those that have delved deeper into the practice of web scraping are already developing strategies. Many (52%) intend to focus on collecting more external data in tandem with other extraction and management strategies, and 50% of the retail and e-commerce organisations polled are looking into ways to collect more real-time data.

Juras Juršėnas, chief operating officer at Oxylabs.io, said: “It is somewhat surprising that such a large proportion of businesses in the retail and e-commerce sectors are still engaged in manual data collection and cleaning. In most cases, third-party web scraping solutions providers offer both extraction and parsing services, greatly reducing the attractiveness of manual data collection, mitigating most implementation costs associated with in-house acquisition, and improving the speed and scope of analysis.”