Are you human? How to defeat the bots in e-commerce
Theatres and other entertainment venues are in a constant war with ticket fraudsters commanding botnet armies, as are other purveyors of high-value, limited-edition goods
Adare SEC produces tickets for, among others, most Premier League football clubs. Its most secure tickets include customised holograms and foils, tiny “microtext” and ultraviolet ink. The aim is to make such tickets very hard to copy – anyone trying may well find the word “void” appearing on their reprints, due to a further security feature.
By comparison, as secure sales director Ian Forster says, a ticket bought online and printed by the customer is on standard 80g A4, making it very easy to copy. For some football games, clubs don’t offer online self-printing because knowing the addresses tickets are sent to helps to reduce those sold to trouble-makers.
But the West End run of hugely popular Broadway musical Hamilton, based on the life of US founding father Alexander Hamilton, goes much further. Buyers are not sent physical tickets, but have to turn up at the Victoria Palace Theatre with the email confirmation, the payment card used and government-issued photo identification, such as a driving licence or passport.
Apart from evoking America through heavy border security, why bother? Because of bots – software that automatically submits purchase requests, giving users huge advantages in buying limited-edition items such as tickets. Some bot users buy for themselves, but they are also used by touts to purchase large quantities for resale.
Delfont Mackintosh Theatres, which owns the Victoria Palace, gives bot prevention as its reason for adopting the paperless ticketing system, run by Ticketmaster. In June 2016, the New York Times calculated that the average mark-up for tickets of the Broadway production of Hamilton on reseller website StubHub was $700 (£500) more than the $172 face price.
However, this system makes life harder for customers, who have to bring valuable documents to a night at the theatre and arrive an hour before the performance starts. It also restricts what they can do with tickets. For example, they cannot be given as gifts if the buyer is not among those attending. While a show like Hamilton can get away with this, a better option is to tackle ticket-buying bots, giving customers who want to attend a show a better chance of getting a ticket.
True fans
Ticketmaster’s answer is an online system that it calls Verified Fan, which it has used for more than 60 tours, starting in the US with musicians such as Bruce Springsteen, Taylor Swift and Depeche Mode, as well as for some New York tickets for Hamilton. The company, a division of US entertainment group Live Nation Entertainment, recently introduced this in the UK for music tours by Paolo Nutini, George Ezra and Jack White.
Carlos Alvarez, senior vice-president for technology at Ticketmaster International, says Verified Fan has an average 90% success rate in keeping tickets from reaching secondary markets. “Fans register in a way that identifies they are a real person – their email address, phone number, or Ticketmaster account, past purchase history – and select the shows they are interested in purchasing tickets to,” he says, before tickets go on sale.
“After the registration period, Ticketmaster uses its proprietary data science technology and an automated and manual process to ensure only real fans participate in the purchase process,” he adds. “Finally, each fan receives a unique code that gives them access to purchase tickets at the stated on-sale time.”
Although this doesn’t guarantee a ticket, it does give customers a better chance, Alvarez adds. “We’re changing the mechanisms of an on-sale so that the speed of bots is no longer a factor in the rush to get tickets.”
Members only
Pre-registering customers is one way to tackle ticket-buying bots. The Royal Shakespeare Company, which opens ticketing for shows in phases through its membership scheme, says it has not had an issue with bots. The Ticket Factory, which sells tickets to events at the National Exhibition Centre complex and has the same owner, runs closed pre-sales for some events at the NEC’s Arena Birmingham concert venue.
“Often, there will be a pre-sale element made available to an existing membership database, and occasionally this can be a ballot style system – with unique codes issued to every customer,” says Ian Smedley, head of application development.
But as with Ticketmaster, The Ticket Factory also handles open sales including for non-NEC venues, so other tactics are required. Smedley reckons it screens out about 90% of unwanted automated bots. “Touts are getting cleverer with their bot software, so agents like ourselves have implemented intelligent software to actively block malicious attacks and fraudulent activity, without stopping genuine fans buying tickets,” he says.
But not all online bots are bad – they include crawler software run by search engines and programs that assist disabled web users. But many bots are used to research availability or buy up tickets, which Smedley describes as “an issue that continues to plague the industry”.
Educate buyers
The Ticket Factory tries to educate buyers to use approved ticket sellers and provides an approved route for secondary sales, he says. “We were the first ticket agent to partner with face value or less resale platform Twickets, as we are against profiteering on the secondary market.” By contrast, Ticketmaster operates secondary market platforms including Get Me In and Seatwave, which have open pricing.
In terms of technical ways to tackle bots, Smedley says a mixture of automation and human checks works well. “Wherever practical, we don’t want anti-bot technology to get in the way of the customer experience for the genuine fans,” he says. “We therefore also use manual checks as support, via our customer service agents. They check the secondary sites on a regular basis to hunt out any suspicious activity involving tickets supplied by TTF, and ultimately can have them removed.” One recent example involved Michael McIntyre tickets.
Alvarez says Ticketmaster spends millions of dollars on anti-bot software, has a dedicated team of developers working on the issue and blocked nearly six billion attempts by bots to access its websites in 2016. But another technical factor involves handling big surges in demand, partly caused by bots. “Each week we have millions of fans flocking to our websites and apps to get their hands on a ticket,” he says. “We experience Black Friday traffic every Friday at Ticketmaster.”
The Ticket Factory experiences the same issue. “We see a surge in traffic for big events, such as Adele or Bruno Mars, and have to balance system resilience with speed of sale and customer experience,” says Smedley. “We use a cloud-scalable queuing system, Queue-IT, to manage customers to our site. The queue contains anti-bot mechanisms such as captcha to ensure only humans get through. Often though, demand far outstrips supply, so the Queue-IT system allows us to message customers in real time.”
Limited editions
Smedley reckons any organisation looking to sell limited-edition goods or services should consider the surge in demand from both bots and humans that their release can trigger. “You need to put your genuine customers at the heart of everything you do,” he says. “Leverage the scale and power of cloud computing to ensure you can cope with the demand to your site – even if you don’t have the transactional throughput that a larger seller like ourselves has.” He adds that it makes sense to tell people when something is sold out, as that should end the demand.
Daniel Smith, head of security research for Radware’s emergency research team, says working out how to handle this surge should be top of the list for anyone planning to sell limited stock. “They should invest in a very good web application firewall,” he says. Such surges can be comparable to a distributed denial of service (DDoS) attack, with 100,000 attempts to access a site by bots and people – and the latter having a very poor experience as a result.
Smith has researched the use of “sneaker bots” to buy limited-edition shoes, with a $220 pair of Adidas V2 Yeezys worth as much as $2,000 on secondary markets. Individuals can buy or rent sneaker bots online – they are easy to find and do not require programming knowledge to use, and a licence typically costs about $300.
Some then rent slots to their friends, allowing them to turn a profit before buying anything. Another technique involves one person programming a bot with the addresses of many numbered apartments in one building to whose mailroom they have access.
Read more about combating bots in e-commerce applications
- AI experts at an EmTech conference in Cambridge, Massachusetts painted a frightening picture of weaponised AI.
- Read more about botnet protection and detection for botnet attacks.
- Expert Ed Tittel explores the features of the top web fraud detection systems and compares critical purchasing criteria.
One answer is to build in challenges that software finds hard to complete. Last year, Nike required those wanting to purchase its limited-edition Momofuku SB Dunk High Pro shoes to use its SNKRS app to scan the menu of a restaurant in New York, either in person or via the restaurant’s website. “It’s two factors – using the right technology, but also making it fun,” says Smith.
However, he is dismissive of captchas, online challenges that require users to click on squares of a picture or retype distorted versions of words. There are bots that can defeat them, he says, adding: “I think we can all say we hate captchas.” Radware uses a range of behavioural algorithms and automated mechanisms for its clients.
Distil, another supplier working in this area, uses the behaviour of users on websites it supports. A human will typically move the cursor around the page in recognisable ways, around specific page elements. Some bots will not move the cursor at all, but “advanced persistent bots” do try to mimic human behaviour.
However, according to the company’s vice-president for outreach and marketing, Reid Tatoris, “there will be movement, but the movement makes no sense”. The system, developed by Are You a Human – a company co-founded by Tatoris and sold to Distil last year – compares these movement to those of previous human users.
Tatoris says older techniques for tackling bots, such as trying to block IP addresses or known bot software, are less useful than they used to be. “We see the average bot exists in the wild for a handful of days,” he says, with the same true of IP addresses. He adds that while captchas can have their uses, they are often over-used and “we certainly don’t recommend turning on captchas all the time”.
While Tatoris believes that organisations selling limited-edition stock should consider paying for specialist services, he says that at a minimum, the pages handling such sales should be set up separately from the rest of the website with security and resilience in mind. “This is going to be a new problem, a new thing you haven’t seen before,” he says. “Understand that this is going to be a threat.”
