Barracuda thought it drove 0-day hackers out of customers’ networks. It was wrong.

In late May, researchers drove out a team of China state hackers who over the previous seven months had exploited a critical vulnerability that gave them backdoors into the networks of a who’s who of sensitive organizations. Barracuda, the security vendor whose Email Security Gateway was being exploited, had deployed a patch starting on May 18, and a few days later, a script was designed to eradicate the hackers, who in some cases had enjoyed backdoor access since the previous October.

But the attackers had other plans. Unbeknownst to Barracuda and researchers at the Mandiant security firm Barracuda brought in to remediate, the hackers commenced major countermoves in the days following Barracuda’s disclosure of the vulnerability on May 20. The hackers tweaked the malware infecting their valued targets to make it more resilient to the Barracuda script. A few days later, the hackers unleashed DepthCharge, a never-before-seen piece of malware they already had on hand, presumably because they had anticipated the takedown Barracuda was attempting.

Preparing for the unexpected

Knowing their most valued victims would install the Barracuda fixes within a matter of days, the hackers, tracked as UNC4841, swept in and mobilized DepthCharge to ensure that newly deployed appliances replacing old, infected ones would reinfect themselves. The well-orchestrated counterattacks speak to the financial resources of the hackers, not to mention their skill and the effectiveness of their TTPs, short for tactics, techniques, and procedures.

“This capability and its deployment suggests that UNC4841 anticipated and was prepared for remediation efforts with tooling and TTPs designed to enable them to persist on high value targets,” Mandiant researchers Austin Larsen, John Palmisano, John Wolfram, Mathew Potaczek, and Michael Raggi wrote in a post Tuesday. “It also suggests that despite this operation's global coverage, it was not opportunistic and that UNC4841 had adequate planning and funding to anticipate and prepare for contingencies that could potentially disrupt their access to target networks.”

The researchers said that at the time they wrote their report, a “limited number of previously impacted victims remain at risk due to this campaign. UNC4841 has shown an interest in a subset of priority victims—it is on these victim’s appliances that additional malware, such as the backdoor DEPTHCHARGE, was deployed to maintain persistence in response to remediation efforts.”

Sometime in October, UNC4841 started exploiting an unusually powerful vulnerability tracked as CVE-2023-2868, which was present in all Barracuda Email Security Gateway appliances sold in years. A flaw in the way gateway appliances parsed logic while processing TAR files provided hackers the all-powerful ability to remotely inject commands directly into the device flow. Better yet, the injection was easy to trigger. By attaching a specially crafted file to an email and sending it to addresses behind the perimeter of a vulnerable ESG device, UNC4841 had a persistent backdoor on hundreds of high-value networks.

Injecting shellcode, courtesy of $f

More technically speaking, the bug resided in the way appliances carried out the qx{} routine in the Perl programming language. It effectively allowed malicious attachments to inject shellcode that the email passed directly into the appliance OS using the user-controlled variable $f. The following ESG code is at the vulnerability epicenter: qx{$tarexec -O -xf $tempdir/parts/$part '$f'};

As the researchers noted earlier, the campaign was already narrowly focused on the most select of targets. According to Mandiant, only about 5 percent of security gateway appliances in existence had been infected. Assuming an estimate from security firm Rapid7 of roughly 11,000 devices (a number Rapid7 said might be inflated) that equates to somewhere from 400 to 500.

Besides DepthCharge, UNC4841 deployed two other pieces of malware in the second wave of their counterattack. One is tracked as SkipJack and the other as FoxTrot or FoxGlove. SkipJack was the most widely deployed of the three. It was a fairly typical backdoor that worked by injecting malicious code into legitimate Barracuda appliance modules. SkipJack was installed on 5.8 percent of infected gateway appliances. Assuming the total number of infected devices was 500 (5 percent of 10,000 devices), the number of those infected devices updated with SkipJack would have been 29. Victims in this group comprised organizations in various levels of government, the military, defense and aerospace, high technology, and telecommunications.

The class of victims infected by SkipJack

Credit: Mandiant

DepthCharge, meanwhile, was used on another 2.64 percent of infected devices, which would be about 13, assuming a total infected base of 500. Targets in this category were US and foreign government entities and high-tech and information technology providers.

Class of victims infected by DepthCharge.

Credit: Mandiant

Backdooring the backup

DepthCharge contained a complex execution chain that was designed to reinfect an appliance as soon as it had been cleaned. It worked by stashing infection code in backup configurations typically used on clean installations of gateways.

The DepthCharge execution chain.

Credit: Mandiant

As the Mandiant researchers explained:

It was common practice for impacted victims to export their configuration from compromised appliances so it could be restored into a clean one. Therefore, if the DEPTHCHARGE trigger was present in the exported configuration, it would effectively enable UNC4841 to infect the clean device with the DEPTHCHARGE backdoor through this execution chain and potentially maintain access even after complete replacement of the appliance. Mandiant and Barracuda Networks identified instances where this may have occurred and notified victims accordingly. Additionally, Mandiant is aware that in some cases, this MySQL configuration database may contain plaintext passwords for user accounts. In these instances, we suspect the actor was harvesting these credentials for lateral movement purposes.

UNC4841’s use of DepthCharge illustrates not just the technical savviness of the group but also its foresight. By stashing away a piece of powerful malware ahead of time, the hackers were able to recoup some of the highest tier of these already carefully selected targets once they believed they had disinfected their devices.

A third piece of malware used in the counterattack is tracked as FoxTrot and FoxGlove. Tuesday’s post didn’t say what percentage of infected appliances received this follow-on attack. The post did, however, say that all the targets who received this payload were government organizations.

Mystery solved, lateral movement, and more

The hackers’ uncanny ability to reinfect their targets solves a mystery from early last month. A June 6 update to the company’s ongoing security advisory no longer recommended patching as a viable means of remediation. Instead, it advised the “immediate replacement of compromised ESG appliances, regardless of patch level.” Until now, the reasoning for the unusual recommendation was unclear.

The post provides additional color on the attack and its targets. The number of countries affected is 86. Government entities made up 27 percent of the known victims, with many located in the United States and Asia. National government entities were the most-targeted group, followed by high-tech and IT organizations and entities in local government. The image below provides a bigger breakdown.

A breakdown of various types of victims.

Credit: Mandiant

The researchers also revealed that UNC4841 members used their access to the gateway appliances and the cleartext credentials they stored to gain entry to other victim resources, including accounts for Outlook Web Access (OWA), SSH apps, and servers that deliver Windows updates. The Mandiant researchers wrote:

In more than one case, Mandiant observed UNC4841 utilizing OWA to attempt to log in to mailboxes for users within the victim organization. In one case, a relatively low number of unsuccessful OWA access attempts resulted in the lockout of a limited number of accounts. In the cases where UNC4841 was able to obtain unauthorized access to a limited number of accounts, Mandiant did not observe UNC4841 send any email from the compromised account. Mandiant assesses that UNC4841 was likely attempting to maintain access to compromised users’ mailboxes to gather information for espionage purposes post Barracuda remediation.

In addition to attempts to move laterally to Active Directory and OWA, Mandiant also observed attempts by UNC4841 to move laterally via SSH to VPNs, Proxy Servers, and other edge appliances on the victims network.

Mandiant also identified accounts created by UNC4841 within the etc/passwd file on roughly five percent of the previously impacted appliances, as another form of remote access. Account names followed a consistent format, containing four (4) randomly generated characters. The actor would then spawn a ssh daemon process to listen on a specific high port and allow login from this newly created user account as another means to maintain backdoor access to compromised appliances. An example of the command is shown as follows:

/usr/sbin/sshd -p 48645 -oAllowUsers=rfvN

In one case, Mandiant identified UNC4841 successfully accessing a Windows Server Update Services (WSUS) server utilizing a domain administrator account identified within the mstore on an ESG appliance. The access to WSUS is notable as Mandiant has observed other China-nexus espionage actors deploying malware on a WSUS server to inject fake updates for remote code execution in efforts to steal data from government entities.

UNC4841 has code and infrastructure overlaps with several known groups working on behalf of the People’s Republic of China, Mandiant said. The researchers indicated that similar campaigns are likely in the future.

“Mandiant assesses that these observations are evidence of the higher level trends we have observed in Chinese cyber espionage and the evolution toward more purposeful, stealthy, and effective operations that avoid detection and complicate attribution,” they wrote. “It is likely that we will continue to observe Chinese cyber espionage operations targeting edge infrastructure with zero-day vulnerabilities and the deployment of malware customized to specific appliance ecosystems.”