Getty Images
Bug bounties not a silver bullet, Katie Moussouris warns
Targeted bug bounties have a role to play in cyber security, but they are not a "silver bullet", and run the risk of wiping out talent pipelines if poorly implemented, warns bug bounty pioneer
Members of the hacker community testified to the US senate about the fragility of the internet in 1998, and little has changed since then, according to Katie Moussouris, CEO, Luta Security.
“We thought in our naivety, when we were invited to the senate and banks, that if we just showed them what the vulnerabilities were, they would be able to fix them and the age of the great internet worms would be over.
“But what we didn’t understand was that deployment strategy required more than just the help of hackers,” she told the 2018 European Cybersecurity Forum in Krakow.
But what has changed, said Moussouris, it that in 2018, Google statistics show that the search term “bug bounty” overtook “penetration testing” for the time, indicating bug bounties are becoming more popular.
“You may have heard that bug bounties are a cheaper way to crowdsource security, and even though I helped create Microsoft’s bug bounty programme and the Hack the Pentagon programme, I am here to tell you that bug bounties are not the solution for the future,” she said.
While vulnerability disclosure is a formal process governed by an ISO standard, Moussouris said bug bounties are an “optional incentive” that may or may not enhance an organisation’s ability to discover vulnerabilities.
“Penetration testing is that lower-risk way to find out about vulnerabilities, and while it lacks that crowdsource appeal, it has with it intrinsically, non-disclosure agreements to minimise risk and let you know what is going on, and it includes professional services.”
Read more about bug bounties
- A lack of best practice guidelines for bug bounties is leaving researchers, organisations and bounty platforms confused and at risk, a report reveals.
- Despite the much-publicised advances in cyber attack techniques, a well-known, well-documented and avoidable vulnerability is still the most popular for attackers to exploit – along with others like it that are often overlooked, white hat hackers reveal.
- Hacker black markets outbid IT companies in bidding for zero-day exploit disclosures.
Cautioning further against relying on bug bounties to find vulnerabilities, Moussouris said that it is not as simple as “opening your doors to hackers” because not all those who respond will necessarily be friendly and helpful.
“Some of the work that I had to do with re-negotiating the Wassenaar Arrangement was all about explaining that there is no ‘technical bright line’ that we can draw around adversarial hacking and hacking for defence or hacking to point out security flaws,” she said, warning of a “number of complexities” in bug bounty programmes, especially around data security.
“The Uber data breach of 57 million records was an example of that. Their bug bounty was capped at $10,000, but Uber paid a hacker $100,000 [to delete the data and keep quiet about]. They incentivised the collection of data against all practical considerations.
“Their actions sent a message to the hackers who were willing to play be the rules, that you can negotiate a higher price by showing you have been able to compromise millions of records instead of just one example,” she said, adding that this and similar actions by other companies is confusing the market.
This state of affairs needs to be fixed, said Moussouris, in a call to action underlined by the threat of artificial intelligence (AI) technologies. “If we are unable to handle the vulnerabilities we know about today, and apply patches for known vulnerabilities, what hope do we have when we enhance the capabilities of bug hunting with AI machine learning,” she asked.
Triage difficult to staff consistently
Reiterating that bug bounties are not a magical fix, Moussouris said that while bug bounty companies promise that they will clean the feed of bugs and mimic the professionalism, safety and risk aversion of a traditional penetration test, they are missing the fact that triage is among the most difficult jobs to staff consistently in information security.
“And even with all the triage support in the world, sometimes there are just too many bugs and the backend infrastructure just can’t handle it,” she said, that one of the reasons Microsoft created its bug bounty programme was because of large volume of bug reports it was getting.
And while the bug bounty programme was able to shape and focus the attention of the hacker community, the volume of reports was just as great after the programme was reduced as it was before.
Moussouris warned that by offering cash rewards for vulnerabilities, bug bounty programmes have resulted in some “perverse incentives”.
“In creating this market we have gotten carried away with incentives instead of looking at the full picture,” she told Computer Weekly.
“Bug bounties have a role to play. As we did at Microsoft, bug bounties that focus incentives on the exact areas you want to discover security issues is a good thing in general, but it is a bad thing if you don’t understand who might have these skills and how you might want to use them in terms of your overall workforce. As a result, it is a bad idea just to seek out individuals with particular skills using high six-figure values,” she said.
Bug bounty payouts
In the past year, Moussouris said there has been a sharp increase in bug bounty payouts. “Top payouts at a quarter of a million dollars for – certainly rare type of finds – but the problem is that you need those people with the rare skills wanting to apply to work inside your companies, not holding out and going for a quarter of a million dollars after the fact because by nature it is too late, especially when it comes to architecture issues.
“We don’t want the next Meltdown and Spectre to be found through bug bounties. We want it to be prevented inside of those chip companies.”
This has a huge potential to damage severely the skills pipeline for organisations that need to be thinking about the architecture for security for the future, said Moussouris.
“Hiring managers inside those companies are worried that with $250,000-incentives on offer, with only a six-month waiting period before former employees of the company can collect them, that they will in fact decimate and gut their own talent pipeine,” she said.
“We have to think that this ‘perverse incentive’ is a real thing, and the offence will always be able to outbid the defence market, so we must not focus on price.”
Virtue signalling
Another concern Moussouris highlighted in her presentation was that paying for bugs has become a form of “virtue signalling” for taking security seriously. “As we look at the future, I would worry about the term ‘bug bounty’ replacing ‘penetration testing’ as we saw in the Google search stats,” she said.
Most bug bounties are good at finding “low-hanging” fruit, said Moussouris, with the majority of bug bounties being paid out for things like cross site scripting vulnerabilities, which suggests very little progress has been made in the past 20 years in secure development to prevent this type of vulnerability.
“One of the things I said to the senate, to give them advice about the future, was that we needed to create more secure coders instead of cranking out people who write the same bugs over and over again,” she said, adding that bug bounties are mainly finding these kinds of bugs that could be found with commercially available tools and even free tools, and not fewer vulnerabilities of greater complexity that teach defenders something new, which was the direction the industry as a whole was supposed to go in.
“When organisations think about starting a bug bounty programme, they are typically not thinking it through. They are not thinking how to prevent vulnerabilities in the place, how they can enhance internal teams to help identify and prevent these issues, and how they can attract more talent to come work inside as opposed to putting out incentives and hoping for the best from the outside.”
In her work with the UK’s National Cyber Security Centre (NCSC), Moussouris said the focus is on building capacity within organisations. “We don’t work on this idea that eventually you are going to offer cash. It’s just not necessary, as proved by the fact that Microsoft’s bug bounty did not increase the number of bug reports they received, they were just more focussed.”
The truth, she said, is that bug bounties are not a replacement for penetration testing, and she warned against “over romanticising” of zero day vulnerabilities and of bug hunting in general, without regard to the entire workforce that needs to be prepared for the future.
“Remember, there needs to be an army behind all security companies to process this information and to be able to do something with it, and once the patches are out, every IT organisation still has to deploy them.”
Exceptional tooling
According to Moussouris, research has shown that one of the most effective things organisations can do is to close the skills gap between defence and offence using “exceptional tooling” that defenders need to assess, at whatever volume, the vulnerabilities that are coming in.
Any organisations that run bug bounty programmes, she said, should think about ways of focussing the attention of the hacking community and putting thresholds in place to cap the number of reports they receive to reduce the signal to noise ratio and ensure they can handle the number of reports they receive.
In closing, Moussouris reiterated the warning that if industry and governments continue to over romanticise bug hunting for cash rewards, they will decimate their workforce of the future. She also reiterated the need to ensure that coders are taught secure development.
“We need to be wary of initiatives that put forward bug hunting as the answer, and instead start looking at the labour market as a whole. And as we look to the future, we need to look for opportunities not to just incentivise destruction, but to build more resilient creators and to support and celebrate the maintainers, instead of calling it ‘the worst job in science’.”
