Track your Black October vulnerabilities from visibility to validated patch

On October 22, Sam Altman said something I haven't been able to shake, “I no longer think we're going to be able to patch all bugs.” He said it as the whole industry was settling on a name for this month, according to The Guardian. Black October.

He's probably right. That's a strange thing to admit when you’ve spent your career working in security and on security products, but look at the last two weeks. Two unauthenticated remote code execution bugs in the PHP interpreter itself. A memory corruption bug in the Brotli decoder that ships inside Node.js. Eight certificate authority keys generated with so little randomness that they could be recovered, all chaining to a root the Python certifi bundle trusted. Then throw in a hijacked LiteLLM release on PyPI and a CVSS 10 GitLab bug, all inside the same few days (the full rundown is here). 

Here's what I think gets missed. Most of the people who'll actually fix this stuff aren't security engineers. It's the developer who owns the checkout service. The incident commander who got paged at 2am because php-fpm workers keep dying. The architect who gets asked “are we exposed?” in a leadership review and has about ten seconds to answer. The engineering manager trying to figure out which sprint work gets bumped. A lot of those folks have New Relic open in a browser tab right now. They just don't think of it as the place to answer that question.

So this one's for you. I'll cover what happened and why it's so painful to patch. Then I'll show you how to find the exposed services in New Relic and, more importantly, how to tell whether the fix has actually landed everywhere.

Key Takeaways

• Black October vulnerabilities target runtime environments below application code: Critical remote code execution (RCE) flaws like PHP2Shell (CVE-2026-109618, CVE-2026-109619) and Node2Shell (CVE-2026-111497) exploit underlying interpreter runtimes rather than direct application dependencies
• New Relic Security RX provides active runtime visibility: By leveraging existing APM agents, Security RX detects actively executing runtime versions and libraries without additional configuration
• Prioritizing production reality and legacy cleanup: Focus remediation on active, running production instances, safely decommission unused services for easy wins along with some cloud cost savings, and systematically verify patch deployment across all remaining live workloads in every environment 
• Validating patch deployment across all running instances: Vulnerabilities remain marked active until every live pod or instance stops reporting the affected version, ensuring unpatched edge cases or failed regional deployments are flagged before security teams close the issue

Black October, in plain terms

The name stuck fast. I can explain the issues in three images. It’s funny and a bit uncanny how there’s always an XKCD to explain everything

For those who have more than a Tiktok’s worth of attention to read, here's the short version of the three disclosures that land closest to where most New Relic users' code runs. 

PHP2Shell (CVE-2026-109618 and CVE-2026-109619)

Two memory-safety flaws in the PHP core, rated CVSS v4 9.2. An unauthenticated web request can reach either one, and both end in remote code execution. On Apache mod_php the exploit fires before any of your application code runs, so your code could be flawless and it wouldn't matter. Wiz Research, who found the bugs with AI-assisted tooling, published the technical writeup. PHP sits behind roughly 69% of websites whose server-side language is known, per W3Techs.

Attackers didn't wait for the patch. Amazon's MadPot honeypots caught the first exploit attempt on October 17, four days before PHP shipped fixes on October 21. Cloudflare CEO Matthew Prince said the network saw thousands of attempts in the ten hours before the announcement and millions shortly after (via SecurityWeek). And just yesterday, October 26, CISA added both CVEs to its Known Exploited Vulnerabilities catalog, as a researcher logged 35 successful shell executions on honeypots running PHP 8.4.26 under mod_php (The Register).

The fixed versions are 8.2.35, 8.3.36, 8.4.27 and 8.5.12. PHP's own advisory is blunt that there are no complete mitigations and updating is the only safe choice. If you're on 8.1 or older, you're on an unsupported branch, so it's a vendor backport or an upgrade.

Node2Shell (CVE-2026-111497)

This one's in the Brotli decoder bundled inside Node.js. If your app decompresses attacker-supplied Brotli data with node:zlib, malformed input can write past the end of an internal buffer. The worst case is remote code execution; the milder outcomes are crashes and leaked process memory. It affects the 22.x, 24.x and 26.x lines, with fixes in 22.23.4, 24.22.1 and 26.12.1. The same October 21 release carried seven more fixes for 24.x, the LTS line a lot of production services sit on. Anything on 20.x or older is end-of-life and won't get a fix.

eMudhra and certifi (GHSA-6q48-6c7w-9gw2)

Eight eMudhra intermediate CAs had private keys generated with too little randomness. They chain to an emSign root that certifi, the CA bundle behind requests and pip, has trusted since 2019. Anyone holding a recovered key can mint a valid-looking certificate for any hostname. Chrome and Firefox pushed blocks quickly. Python apps and containers carry their own CA bundle, though, so they stay exposed until someone upgrades certifi to 2026.10.19 or later (GitHub advisory). One gotcha: pip vendors its own copy of certifi, so you need pip 26.2.2 as well (Python.org guidance). eMudhra has since announced it's leaving the TLS certificate business entirely.

Why is patching this so hard?

Picture a normal patch week on your team. Now drop a dozen new CVEs on it at once, a couple of them critical, in the parts of the stack nobody thinks about. That is what’s happening right now for a lot of teams.

There's simply more of it every year, and 2026 broke the curve. We'd already passed all of last year's CVE count before the end of September. Nobody's team grew that fast.

Then there’s the question of how quickly you can find these across all your services. They ride in a base image someone picked up a while ago and so you need to patch there and then deploy across every instance. Until that happens you’re still serving traffic on a vulnerable interpreter. Just figuring out where you’re affected, who owns the service, and which base image it comes from takes longer than it should. 

While you’re figuring that out the clock’s ticking. With PHP2Shell, the first exploit attempt showed up four days before a fix even existed. Within a day of the patch, Cloudflare was counting attempts in the millions. Most teams, on a good day, measure remediation in weeks.

Say you move fast anyway and ship the fix. There’s still one more catch…rebuilding the image turns the scans green but the new version has to reach every instance in every region, at the same time every other team is pushing for rebuilds as well. It only takes one missed instance in the deployment to cause a breach.

If you want the deeper numbers, the Verizon 2026 DBIR and Black Duck's 2026 OSSRA report are both worth an afternoon.

Which brings me to the question I think matters most this week: is the fix running everywhere yet? Most scans can't answer that one.

Where New Relic helps

Most vulnerability tools work from what your code and images say should be running. That's useful, right up until you're scrolling through a few hundred findings for packages that never even load, or a dashboard swears everything's patched while one pod is still happily serving traffic on the old version.

Security RX comes at it from the other end. It builds on the APM agents you already run, so it sees what's  actually loaded and running in each service, with no changes to your agents, new tools to roll out, or even upgrades to handle.

Less noise, including the stuff under your code. Security RX only shows the library modules that are actually loaded in your running services, plus the runtime environment itself, like the PHP or Node.js version. So PHP2Shell and Node2Shell show up right alongside your library findings instead of in a separate report nobody on your team has a login for. Each finding gets a priority and a Reason to prioritize, like Exploit extremely probable.

“Fixed” means fixed everywhere. Plenty of tools close the finding the moment the PR merges. Lovely, if your deployments always go to plan. Security RX won't mark a CVE resolved until it's gone from every running instance, so you catch the rollout that's still crawling or the region it skipped. It’s a rough week to be in security already. Give your CISO some assurance so they can get some sleep tonight. 

Here's an example similar to what came up this week already. Say your payment service shows up as affected by the Brotli RCE (CVE-2026-111497) because it's running Node.js 22.14.0, and the owning team swears they moved to 22.23.4 yesterday. Everyone's a little annoyed. Is the tool wrong? Open the CVE and dig into the details for that service. Odds are one instance never got the memo and is still running 22.14.0 while the rest show No longer detected. The team did upgrade, and the tool's right too. You've just found the one instance the deploy missed.

One caveat: APM agents report runtime environment data every 30 minutes by default, so a freshly deployed service can take a little while to flip to No longer detected. To be honest, I'll take a short delay over a blind spot any day.

Better together. Scanners are great at catching problems before they ship and showing you everything that could be exposed. Runtime tells you what's actually exposed right now and whether the fix made it out. You can use one over the, other but you’ll be better off with both.

How to track Black October in New Relic

Here's something we've noticed about how people actually use Security RX: it's part of everyday engineering work. Most of the time, an engineer opens it while they're already investigating a CVE within a service they own. In over half of those sessions they open the side panel to give security in context like errors, logs or traces, usually within a minute, to quickly triage. Then they come back while the fix deploys, watching for any post deployment issues and regressions. That's security and software maintenance all wrapped together in context.

Let's go through a few questions you may need to answer and how to do that in New Relic now.

Answering “are we exposed?” for the whole org

For the architect or engineering leader who needs the big picture, go to Security RX > All Vulnerabilities and set the Disclosed at filter to the last 1 to 7 days. Everything recently disclosed that touches your estate shows up with a priority, a Reason to prioritize, and counts of affected and No longer affected entities. Remember you can use Entity and Team filters here to get more granular details like what you own that's running in production. Paste CVE-2026-111497 into the Search by name or cve box to jump straight to one.

Alternatively you can start from the Overview > CVE Advisory  that will give you a prefiltered list or All Vulnerabilities page as well scoped to a specific account like Production.

Triage from the service you're already in or quickly filter to see it across your enterprise 

1. Start from your service. Every APM service summary has a Vulnerabilities tile with critical and high counts, sitting right next to Issues, Deployments and Service levels. Click through, or use the Security RX section in the left nav (Overview, Libraries, Vulnerabilities). Filter on Time since first detected for the last week to see only what's new.

2. Open the CVE. The detail page has remediation instructions with the fixed upstream and distro package versions, the vulnerability activity log and any linked Jira tickets. The side panel keeps the service's own signals in view: response time, throughput, error rate, deviations and logs.

3. Get the context before you decide. This is the moment most people hop over to the service's errors, logs and traces anyway. Is the service already acting strange? Are workers restarting, or is something spawning child processes it shouldn't? (The watch list below has more.) A quiet internal service and a public checkout API that's suddenly throwing errors deserve very different levels of urgency.

4. Decide what dead weight, what can wait, and track it. The fastest way to bulk close a ton of vulnerabilities and save some cloud cost is to shutdown unused services. Then maybe you have a service that never handles Brotli data from outside and you've decided to patch it next sprint. Use Change to ignored and set to 14-days, a reason, a description and a documentation link based on your security exception handling guidelines. Every change lands in an audit log and it will land back on your list for next sprint planning.  

Watch the fix land

1. Keep an eye on errors while it deploys. The CVE page that told you what to upgrade also shows whether error rate and response time move once the patch ships. Nobody wants to trade a vulnerability for an outage.
2. Confirm it's gone everywhere. Click into the CVE's full impact (or See all entities) to get Area of impact and Remediation status. The chart shows affected entities dropping and No longer detected climbing over time. Use the Status dropdown to filter to what's still affected, and the Reporting dropdown to include entities that are offline right now, so a host that's down for maintenance doesn't quietly fall off your radar.

Things worth watching while the patches roll out

Patching is the real fix here, full stop. PHP said there's no complete mitigation, and Node.js didn't publish a workaround. While you work through it, a few signals in the telemetry you already collect deserve a closer look:

•  php-fpm or node workers crashing or restarting more than usual. Memory corruption attempts tend to look like instability before they look like anything else.
• Out-of-memory errors or “zend_mm_heap corrupted” messages in PHP logs. OOM handling is exactly the path CVE-2026-109618 abuses.
•  Bursts of big, malformed multipart POST requests to PHP endpoints, often with huge numbers of fields or parts.
• Child processes spawning from php-fpm or node, especially shells or curl. That one's a strong sign somebody already has code running.
• Unusual outbound traffic from PHP or Node.js hosts.

None of this replaces your security team's tooling. It's just easier to notice the weird stuff when you're already staring at the service. And if you truly can't patch today, PHP's advisory lists stopgaps like disabling session upload progress and tightening request size limits. Treat them as a bridge to the upgrade.

Happy Halloween, I guess

Halloween is Saturday, and I'd bet a fair number of you will spend part of it watching a rollout crawl across regions. If that's you, keep the Area of impact page open in a tab. And go easy on whoever owns that last instance still on the old version and the candy corn. No one knows why either really still exists.