Microsoft finds Storm-0558 exploited crash dump to steal signing key

Microsoft has published details of how a Chinese state-backed advanced persistent threat (APT) actor tracked as Storm-0558 was able to get its hands on a Microsoft account (MSA) consumer key to forge tokens with which they accessed OWA and Outlook.com accounts at multiple high-profile customers.

The incident, which was first disclosed on Tuesday 11 July, began in April, and enabled Storm-0558 to access email account data across multiple organisations, including US government agencies and departments, from mid-May. It took another month for the intrusions to be detected and dealt with.

Microsoft has since faced criticism for its response to the incident, including accusations of negligence from government officials.

Redmond’s investigation into the incident has now concluded, through which Microsoft discovered that Storm-0558 was able to take advantage of a rare confluence of events to begin its campaign of cyber attacks. It has taken action to prevent this situation from arising again.

“Microsoft is continuously hardening systems as part of our defence-in-depth strategy. Investments which have been made related to MSA key management are covered in the https://aka.ms/storm-0558 blog. Items detailed in this blog are a subset of these overall investments,” said the firm’s MSRC security team.

Because of the highly sensitive nature of its customers’ data, Microsoft maintains highly controlled, isolated and restricted production environments that are designed to prevent key material from leaving.

However, in April, a consumer signing system crash led to a snapshot of the crashed process – this is known as a crash dump, a type of memory dump. Such crash dumps redact sensitive information and this one should not have included the signing key, however, in this case, the signing key was included due to a race condition – an issue that occurs when two program processes or threads attempt to access the same resource at the same time, causing problems in the system. Microsoft’s systems did not detect this at the time – something it has since addressed.

Because Microsoft did not believe the crash dump contained sensitive material, it was moved from the isolated production network into its debugging environment, which is on the main, internet-connected corporate network, consistent with Microsoft’s standard debugging processes. The key’s presence was still undetected at this point.

At some point following this, Microsoft found, Storm-0558 successfully compromised a corporate account belonging to a Microsoft engineer with access to the debugging environment. Unfortunately, Microsoft’s log retention policies mean there is no direct evidence that Storm-0558 exfiltrated the signing key from the debugging environment at this point, but this is by some margin the most likely mechanism used.

An already bad situation was made worse for Microsoft and its government customers because, to meet growing demand to support applications that work with both consumer and enterprise applications, Microsoft has had a common key metadata publishing endpoint for the past five years.

When setting up this converged offering, Microsoft said it did update documentation to clarify the requirements for key scope validation – that is to say which key to use for consumer accounts and which for enterprise ones. It also provided an application programming interface (API) via its library of documentation and helper APIs to help validate signatures cryptographically, but critically, did not update these libraries to perform scope validation automatically, something else it has now moved to correct.

When its mail systems were updated to use the common metadata endpoint in 2022, developers incorrectly assumed these libraries performed complete signing key validation, and as such, didn’t add the required issuer/scope validation, leading to a set of circumstances whereby the mail system could happily accept a request for enterprise email using a security token signed with the consumer key.

“Crash dumps do not usually contain signing keys, but this was not the only mishap that generated the blunder,” said Jake Moore, global cyber security advisor at ESET.

“As with a vast majority of attacks, threat actors were able to compromise an engineer’s ‘access all areas’ corporate account which successfully began the attack, so it is vital that such accounts remain on high alert and vigilant to phishing attacks,” he added.

“This key then essentially powerfully mimicked any user account or application they desired. Acquiring the key from a previous crash dump which enabled the hack into Microsoft cloud services was possible as credential scanning was not previously accounted for.”