Log4Shell on its way to becoming ‘endemic’

The Log4Shell vulnerability in Apache Log4j, which caused consternation across the technology industry when it surfaced at the end of 2021, will be with us for a long time to come, perhaps as long as a decade, according to a report produced by the US’s Cyber Safety Review Board (CSRB), a panel of experts drawn from various government agencies and the private sector.

The CSRB, which was established by president Joe Biden via an executive order in 2021, has been poring over exactly what happened with Log4j, a pervasive and ubiquitous Java-based logging library that has been incorporated into thousands of systems over the years.

Tracked as CVE-2021-44228, the Log4Shell remote code execution (RCE) vulnerability is considered very easy to exploit and has been described variously as a “design failure of catastrophic proportions” and a “worst-case scenario”.

Now, more than six months on, it has become abundantly clear that despite the urgency with which the industry stepped up to address it, Log4Shell is by no means over, as was made clear by CSRB chair Robert Silver, under secretary for policy at the Department for Homeland Security, and deputy chair Heather Adkins, senior director of security engineering at Google.

“Log4j remains deeply embedded in systems, and even within the short period available for our review, community stakeholders have identified new compromises, new threat actors and new learnings,” wrote Silver and Adkins. “We must remain vigilant against the risks associated with this vulnerability, and apply the best practices described in this review.

“The Board assesses that Log4j is an ‘endemic vulnerability’ and that vulnerable instances of Log4j will remain in systems for many years to come, perhaps a decade or longer. Significant risk remains.”

At the time of writing, the CSRB said it was not aware of any significant attacks on critical national infrastructure (CNI) that have exploited Log4Shell, and also noted that general exploitation occurred at lower levels than was at first predicted, given its severity.

However, it further noted that these conclusions were not easy to draw because much of the evidence is anecdotal and there are no real sources to understand exploitation trends across geographies and industries that are not linked to commercial cyber interests.

The CSRB further assessed that in the response to Log4Shell, many things went right, particularly in the Apache Software Foundation’s (ASF’s) response, which quickly recognised the severity of the vulnerability and was able to fall back on well-established software development processes to remediate it. The CSRB also praised the response of the cyber industry in general, with vendors quick to produce guidance.

Yet, it added, organisations clearly struggled to respond to the event, with much of the hard work of upgrading vulnerable systems still far from complete. Also, Log4Shell exposed troubling security risks inherent to the open source community, which the report said was inadequately resourced to ensure that code is developed in accordance with security best practice.

The report went on to make 19 key recommendations, which are broken into four categories, set out below, while the full report can be downloaded for review here.

Terry Olaes, sales engineering director at Skybox Security, a California-based threat management specialist, has been tracking Log4Shell since the vulnerability was first disclosed in December 2021. He described the report’s findings as unfortunate, but not surprising given Log4j’s widespread use.

“Log4j threats expose victims that lack mature cyber security risk models to attacks that have RCE vectors like ransomware, and there will likely be many attacks associated with this vulnerability for years to come,” said Olaes. 

“In the years ahead, threat actors will innovate new and creative ways to exploit common tools like Log4j. As a result, preventing breaches requires immediately minimising your exposure through smart and targeted mitigation.

“For a widespread vulnerability like Log4j, patching all of the instances isn’t practical. Not only is it time-consuming, it’s also hugely costly. History shows that the ‘patch everything’ strategy is a monumental waste of effort due to the fact that, typically, it’s a very small subset of devices that are actually exposed to the attack itself. That is why it is crucial to take a more proactive approach to vulnerability management by learning to identify and prioritise exposed vulnerabilities across the entire threat landscape.”

Commenting further on the CSRB review, Synopsys Cybersecurity Research Centre principal strategist Tim Mackey said: “Rarely do we get a comprehensive review of the impact and root causes of a cyber incident so quickly after the incident occurred, but that is precisely what we have from the CSRB in their report on Log4Shell and Log4j.

“Open source software is fundamentally managed differently than commercial software, but open source software plays a key role in the success of commercial software. The long-tail scenario outlined in the report is one we’ve seen with countless past vulnerabilities, and one that favours attackers since their success is based on having at least one victim who hasn’t patched their systems.

“Given that management of open source software is different than commercial software, and open source powers commercial software, reliance on a commercial vendor to alert consumers of a problem presumes that the vendor is properly managing their usage of open source and that they are able to identify and alert all users of their impacted software – even if support for that software has ended.

Mackey added: “With patch management being a challenge at the best of times, to mitigate the risk of unknown open source governance within vendors, software consumers should implement a trust-but-verify model to validate whether the software they’re given doesn’t contain unpatched vulnerabilities.”