ButterCMS is a popular tool used to manage content for blogs. Earlier this week, we noticed a potentially severe security incident which triggered the team to remove ButterCMS from our site, and start an in depth investigation into what happened. Potentially 1.660 websites and over 5.800 domains were impacted.
Our aim is to share the findings of our investigation to show what can happen when you trust dynamic 3rd parties without continuous verification.
The ButterCMS incident
We observed the incident beginning at 08:00 (PT) September 9th, when we noticed a significant increase in errors because the DNS was failing to resolve the hostname. This resulted in an outage of the blog on our website.
Shortly after, we noticed the ButterCMS site was down because no DNS records were being served.
When we dug deeper, we noticed the ButterCMS domain had a WhoIs update at the same time as our issues started. A logical reason for this, could’ve been a renewal or a change in ownership of the domain. The latter would be highly concerning.
If the domain was “sniped’ and had fallen into malicious hands, it could have posed a serious security risk, akin to the Polyfill incident, where a change of domain ownership caused a major browser supply chain attack on nearly 500.000 websites. This resulted in malicious code being injected in millions of visitors' browsers resulting in malicious redirects and potentially other stealthy attacks which were not noticed due to client-side security monitoring being under utilized.
By now, we disabled the ButterCMS integration entirely to prevent any data from being served from the potentially compromised domain. Disabling the CMS removes the risk of malicious code being injected into our site.
We reached out via X (Twitter) to notify the ButterCMS team:
@ButterCMS We're experiencing DNS issues with https://t.co/tnhJP0PZWG, it seems like your DNS was entirely wiped on several different networks.
Since I can not access the website I have no other way to contact your support team.— cody (@devlooskie) September 9, 2024
They did not respond to our message.
At 08:30, we observed the DNS records for ButterCMS were being restored, and importantly, pointed to the same IPs as before the downtime. Suggesting that the issue was likely due to a misconfiguration or downtime with the DNS provider used for their domain. During the downtime, we saw no records present. Not for the site, nor the API.
Once we confirmed the service was back to normal, we reverted our changes.
We finally reached out to the ButterCMS team via their chat service:
In the full message, the support operator claimed her team was unaware of the issue but started investigating. We can’t share the full message history, as after asking for a case number, they responded they didn’t have one. Intercom should have these by default.
We checked the ButterCMS status page, which to date, shows no downtime. This initially made us believe the issue was either on our end, or larger than it turned out to be.
The failed correspondence with ButterCMS
After all of this, we contacted them via email. Here was their response:
They mention a recent acquisition of ButterCMS by Tiugo Technologies. Yet this was reported late in 2022, over 1.5 years ago. While this now shows the reason for the domain transfer, we were not aware or notified prior to the changes.
We followed up, and received the following message:
A verification issue could’ve been the problem. We’ll know more in their post-mortem statement to be posted soon.
In a final followup email on the 12th of September, this was their response:
We believe communication about these types of changes are vital. When planned changes go wrong, even slightly, it puts customers at significant risk. A quick email to notify users goes a long way in these types of scenarios.
These changes could even interfere with GDPR requirements, now knowing the domain ownership changed after an acquisition. GDPR doesn't specifically mention domain transfers but focuses on the transfer of control over personal data. If the change in ownership leads to a new data controller managing personal data, the customers (data subjects) would need to be informed. This falls under GDPR's principles of transparency and fairness (Articles 13, 14).
Customers must be notified about the new controller's identity and any changes in how their data will be processed. The notification must occur within a reasonable time frame.
We waited to post this for some time to see if any communication might yet come prior to the planned webinar. Thus far, we haven't seen any.
UPDATE: Even after the webinar, no recording and no blog post or statement.
Later we tried looking for other companies commenting on this issue, but found none. While we can't know their exact numbers of customers, potentially 1.660 websites and over 5.800 additional domains were impacted:
The risk with dynamic 3rd parties
This is more serious than simply records changing. This pattern is exactly what happens when the domain changes ownership, or major WhoIs details are updated.
Although the issue is now resolved, this situation showed a potential security vulnerability.
The broader lesson from this incident is the risk you take when relying on third-party services that dynamically inject content into websites. A problem we know well, and protect against on the client-side.
Domains used in integrations, can be bought by malicious actors and exploited for attacks. The content they inject is dynamic. Meaning the content can change based on various factors like time, region, and other user-specific data making it easy for bad actors to circumvent detection.
In this case, the renewal of the ButterCMS domain, and the lack of clarity around the WhoIs update, raised a red flag to remind us to monitor third-party dependencies.
How a website is built significantly affects its vulnerability to this kind of issue. Ideally any input should be properly sanitized. User input should be cleaned to prevent malicious code from being injected into the site.
Many developers do not implement proper sanitization, especially when it isn’t explicitly addressed in documentation.
If you search for "dangerouslySetInnerHTML," there isn’t clear guidance on how to safely use this prop. Without proper sanitization, any valid HTML can be injected, which creates a serious XSS (Cross-Site Scripting) vulnerability.
When HTML is injected into the client, the entire webpage can become compromised through script injections. Without safeguards, the injected HTML could execute harmful scripts or redirect users to malicious sites, effectively turning the feature into an open portal for security risks.
How this is dealt with client-side
Of course domain change risk is not new to us. It’s a common attack vector in the client-side security world. c/side already checks DNS record history, WhoIs information and metadata.
As long as the website stays up (and as a result, our proxy does too) and any changes or anomalies occur, our engine detects this. And, after checking other possible malicious activities, will alert and/or block the domain, the script, and thus a possible attack.
You can protect your site quickly and for free by using c/side.