The cost of false positives - how we became a target

This week, we identified an intriguing use case involving the WP3[.]XYZ attack (link to our blog post). It sparked interest across the community and led to better detection rates on platforms like VirusTotal (VirusTotal link).

While most appreciated our efforts, others criticized us for not identifying the root cause or recommending services to clean up hacked websites. Despite this, we aim to make the community aware of potential attacks and promise to do even better in the future.

When false positives hit home

After publishing the blog, something unexpected happened: our main website, cside.dev, was flagged as suspicious. This incident is significant because it highlights how a simple false positive can disrupt not just technical workflows but also the reputation and operations of a business.

For technical readers, it underscores the importance of accurate detection rules, while for non-technical readers, it demonstrates how such issues can escalate and impact trust and business continuity.

Naturally, we were caught off guard and immediately began investigating.

Upon closer inspection, it seemed that some lazy detection rules might have caused the issue.

I’ve worked in endpoint security for over four years before transitioning to a SOC (Security Operations Center) analyst role. This experience gave me firsthand insight into the challenges of managing alerts and detections—a topic closely tied to the real cost of false positives.

As someone who has written detection rules, I understand the temptation to take shortcuts. Early in my career, I made similar mistakes. With experience, I learned the difference between good and bad rules and the importance of leveraging the right technologies for rule writing. This is why most security vendors employ multiple engines to address different types of attacks effectively.

In this case, the issue seemed to arise from two factors:

1. The WP3[.]XYZ subdomain: We use a service to generate brief descriptions of domains using internal data and AI (c/side Domain Insights). While the descriptions aim to provide useful context, they are not intended for detection purposes.
2. Misinterpretation by AV vendors: Some antivirus vendors flagged the domain name simply because it appeared in our URL. This kind of detection, based solely on the URL structure, lacks context and can lead to unnecessary disruptions.

To help the community, we shared the complete code in our blog for other companies to improve their detections. However, flagging a domain serving malicious payloads without proper context is a problematic practice. We have addressed this issue in detail in another blog post (Are Threat Feeds Still Good in 2024?).

The real cost of false positives

False positives may seem insignificant at first glance, but their impact can be profound:

1. Operational disruption: Even a 1:100,000 false positive rate can have severe consequences if it disrupts critical transactions. For example, imagine a payment gateway being flagged incorrectly during a peak shopping season. This could block thousands of legitimate transactions, resulting in frustrated customers, lost revenue, and potential damage to the company’s reputation.
2. SOC analyst fatigue: FPs can waste thousands of hours as analysts attempt to triage and investigate non-issues. This fatigue can lead to true positives (TPs) slipping through unnoticed.
3. Business impact: This is the most concerning consequence, as it can jeopardize a company’s entire operation.

Let’s dive deeper into point three.

At c/side, we are a small, young startup, and our journey has been fueled by passion and a desire to contribute meaningfully to the cybersecurity community. However, if security products flag our website as malicious, it could jeopardize everything we’ve worked for.

I remember the initial panic of seeing the detection—it was not just about fixing an issue, but about safeguarding the trust we’ve built with our customers. The time and energy spent addressing such problems could disrupt operations significantly, especially for small companies like ours. While large corporations might weather such challenges, for smaller organizations, it could spell disaster.

Why it matters

The real cost of false positives goes beyond technical challenges. It impacts businesses, customers, and livelihoods. Those who have experienced the repercussions firsthand know how deeply it hurts.

Our commitment at c/side

At c/side, we firmly believe in a zero false positive policy.

We strive to:

• Share whatever we know with the community openly and transparently.
• Use our own products to ensure accuracy and reliability.
• Minimize the impact of false positives while enhancing our detection capabilities.

We understand the cost of both false positives and cyberattacks, which is why we’re committed to continuous improvement and collaboration with the broader community.

If you have questions or suggestions, feel free to reach out. We’re always open to feedback to make our efforts even better.