Common Vulnerabilities and Exposures or CVEs is a catalog of publicly disclosed software security flaws that attackers can use to gain unauthorized access.
With 40,003 CVEs recorded in 2024, denoting a 39% increase as compared to 2023, container vulnerabilities are no longer a minor cleanup task. It is a river that floods Kubernetes clusters, drowning the teams in operational burden while real exposures slip through.
Chasing every single CVE is a road to nowhere as new vulnerabilities are discovered constantly. The solution is to build a centralized vulnerability management system that will serve as a dam resisting the CVE flood and enable controlled CVE identification and remediation.
This article lays out practical approaches to establishing such a system based on hardened images, provenance, SLAs, consolidated reporting, and clear update and remediation policies.
The Endless CVE Flow and Why Kubernetes Feels Like a Battlefield
Container images have revolutionized the field of cloud-native computing, offering a faster, easier, and cheaper way to deploy services as opposed to virtual machines. Kubernetes, in turn, scaled that model by orchestrating fleets of containers. So far so good.
And yet, for many teams Kubernetes is not a safe harbour anymore. Instead, it turned container operations into a battlefield, where every scanning result returns hundreds of CVEs.
Here’s a realistic situation. A vulnerability dashboard lights up with 1,000 CVEs across your container estate. Tickets multiply, release gets postponed, and engineers spend a whole week patching third-party code they didn’t write. Next release, the situation repeats. Eventually, the noise teaches the engineers to ignore alerts.
This isn’t because the engineers are doing their job badly or vulnerability scanners are broken. It’s because container images contain many layers: OS packages, language runtime (Java, Node.js, etc.), system libraries and dependencies, configuration files. Each layer can add an attack path: outdated packages, unpatched dependencies, base images from untrusted sources, misconfigurations. What is worse, software ages fast. What was secure yesterday becomes vulnerable today.
That churn never stops. As a result, what was clean at build time becomes vulnerable at later stages without a single line of code changing. The survival mode becomes a routine.
But it doesn’t mean that the production containers should be left riddled with vulnerabilities. CVEs need remediation. The scalable goal, though, is not perfection (zero CVEs), but control. The control implies zero unmanaged risk. You always know
- What’s running,
- Which findings are relevant to running workloads,
- Who owns remediation,
- What the SLA for patches is.
That requires a mindset shift in metrics. One cannot realistically enforce zero CVEs in production. What one can enforce, however, is zero untracked artifacts, zero unsigned deployments, and zero exploitable vulnerabilities lingering past SLA in critical workloads, such as internet-facing ones or handling sensitive data.
Once you set out on operating this way and start triaging the findings, the following question becomes obvious:
What is producing the most CVEs, and why is it so repetitive?
In most organizations, the answer is: ungoverned base images.
Hardened Images: A Highest-Leverage Way to Shrink the CVE Flow
A base image is literally the foundation of a container image. It includes the OS, the runtime if necessary, and required packages for the application to run. In enterprise development, teams usually use a variety of base images specific to the container image they are deploying.
If the teams build workloads on random base images, they are inviting vulnerability chaos to production. Every team pulls a different base, a different version, a different set of packages, and implements a different patch cadence. The result is inconsistent security posture and a patching workload that never drains.
The easiest way to stop the flood is to standardize what everyone starts from, which is exactly what hardened base images do.They drastically reduce the number of inherited vulnerabilities and centralize the work required to fix the remaining ones.
But what exactly does the concept imply?
Hardened images combine minimal attack surface with proactive security, built-in provenance, and vendor accountability. Let’s look into these concepts in more detail.
Minimal attack surface
A base image has low-to-zero CVEs by design and an immutable set of only those components required by your application to run. Thanks to that, the production containers will be free from known vulnerabilities from the start and cannot be changed or inspected externally.
Built-in provenance
A verifiable record of where a software component came from and how it was built, typically, a Software Bill of Materials (SBOM), listing all components used to build a software artifact plus versions, and a cryptographic signature verifying that the software artifact wasn’t altered. Provenance data equips the teams with verifiable proof of a software component’s origin and is indispensable for compliance and audits.
Vendor accountability
A hardened image isn’t just a single artifact you download and forget. It is part of a curated family of base images with shared properties: the same hardening baseline, provenance guarantees, and commitment to regular patching across variants. For that to stay true over time, someone has to take responsibility for maintaining that family. That’s where vendor accountability enters the picture.It means that the vendor owns the base images as a product, offering a defined release cadence and response SLAs when critical issues emerge. That creates a stable patch rhythm that application teams can rely on and enables tech stack unification.
There are several vendors providing hardened base images, including BellSoft, Chainguard, and Docker. When selecting a vendor, look for one with OS security and compliance expertise, strong supply-chain guarantees with signatures, attestations, and SBOMs, and a predictable patch cadence backed by an SLA.
In addition, verify how the image is built. The best providers build the OS and runtimes from source so you’re not inheriting mystery third-party binaries with unknown CVEs and chaotic update schedules.
Migrating to hardened container images is a big leap forward. You will trade an unmanaged pile of ad-hoc base layers for a controlled, consistent, and auditable foundation. With that groundwork in place, you can design a vulnerability management system with enforced policies on what can be pulled, continuous rescans, and automated rebuilds.
In the next section, we’ll build that system step by step.
From Hardened Base to Governed Vulnerability Management
Hardened base images guarantee low-to-zero CVEs provided that they are updated regularly, but your estate also includes the application dependencies, and the orchestration platform. Any software artifact can bear vulnerabilities.The goal remains the same: zero unmanaged risk.
Here’s how to achieve that.
Standardize on Chosen HARDENED Images
You cannot build a reliable vulnerability management system without standardization.
Standardization means that an enterprise uses a small set of hardened base images selected by the platform team. These images are enforced by the build rules. This way, you can easily prove what’s running in production and automate rebuild and rollout when the base images get updated.
Enforce Build Immutability with Pin-by-Digest
Hardened base images are immutable, but the build process may be not. This is why standardization should go hand-in-hand with determinism.
Deterministic build process means that given the same input, the build produces the same output down to the byte. This enables the teams to ship exactly the software they reviewed, which, in turn, makes it easier to detect changes between runs, whether it’s tampering, a dependency update, or an upstream release that swapped artifacts
The best way to achieve determinism is to pin everything:
- Toolchain: compiler, language runtime, package manager
- Build system: container images, CI runners, plugins
- OS and system dependencies
- Base images
Base images are usually pinned by tag, i.e., a label used to identify the software component. Tags are convenient, but they also can drift over time, which will result in inconsistency between what you intend to run and what you actually run in production. Therefore, the best practice is to pin base images by digest.
Once the build determinism is established, the system becomes reliable and auditable.
Generate an SBOM and Provenance Data with Every Build
CVE scanning without the inventory produces a lot of noise or leaves critical vulnerabilities unnoticed. A Software Bill of Materials (SBOM) solves this issue by providing the data on the libraries and modules used for building a given piece of software. Every build should generate an SBOM that lists the components inside the image.
An SBOM can be stored in artifact repositories together with the application binaries, they can be attached to container images in artifact registries, or they can be stored in a dedicated SBOM management platform such as Dependency Track. The key is to make them versioned, traceable, and accessible to audits and scanning.
Provenance goes one step further by tying the software artifact to the build process that produced it. Provenance data includes, among others, builder identity, source repo commit, and build steps.
SBOMs and provenance are demanded by several regulations, so you must implement them to pass audits successfully. However, they do more than satisfying compliance requirements:
- When a new CVE appears, you can quickly assess the scope of impact.
- You can cleanly split responsibility: what’s inherited from the base image versus what the application added.
Sign and Verify Artifacts at Deploy Time
After collecting the SBOM and provenance data, the images must be cryptographically signed to ensure their integrity and trustworthiness. Signing images and their attestation helps to prove the authenticity of an artifact and gives protection against deployment of malicious software. On the other hand, verifying the signatures at deploy time enables policy enforcement: the cluster accepts only artifacts produced by your pipeline, built from approved bases, and with attached SBOM/provenance.
Therefore, the total workflow is as follows:
- Generate an SBOM and provenance data for every image.
- Sign and store an SBOM and provenance data as attestations tied to the image digest. For example, an open source tool Syft can generate an SBOM and create an in-toto attestation signed with Cosign. The attestation is stored as a JSON file and can be attached to a container image in a registry.
- At deploy time, verify signature and provenance.
After that, three scenarios are possible depending on the verification outcome:
- Accept: signature and attestations are valid, images can be deployed.
- Quarantine: anomalies that necessitate review, such as unverifiable signer, outdated SBOM, or medium-severity issues with compensating controls.
- Reject: non-compliance or high risk, for instance, unsigned image, wrong identity/issuer, missing or forged attestations, mismatched source/ref, critical vulnerabilities with available patches not applied.
Introduce Continuous Scanning
Scanning the images once at build time is not enough. New CVEs emerge constantly, and some risks are not related to CVEs: excessive capabilities, elevated permissions, exposed secrets, and so on.
Therefore, you need to establish a continuous scanning loop to identify and remediate security risks promptly in existing artifacts and running workloads.
Scanning should cover
- CVEs in base images and application dependencies,
- Configurations and risky defaults such as the absence of resource limits or seccomp,
- Suspicious runtime behavior such as suspicious syscalls.
Once you have established what to scan, the next question arises — when to scan. Scanners vary in purpose, so you can implement different solutions at different stages of the software lifecycle:
- On every build in CI to stop critical flaws such as exploitable vulnerabilities or missing provenance/SBOMs early;
- On image push to validate attestations and signatures;
- Periodically against the SBOMs to catch newly disclosed CVEs affecting the versions you already run;
- Continuously in runtime to detect compromising and suspicious behavior.
But the scanner findings without a clear triage plan is just noise. The risk model that will help to prioritize scanning outcomes may look like that:
- Which component does a vulnerability affect?
- Is the vulnerability exploitable?
- Does a patch exist?
- Is the vulnerable component exposed externally?
- What is the blast radius?
Then, the findings can be conveniently mapped to the triage scenarios: patch now, patch with the next update, create an exception with expiration date.
Below is a an example CVE reporting template:
- Workload or service;
- Owner team;
- Risk tier;
- Why it matters: exposure, exploitability, privileges;
- Fix path:
- Rebuild against Base X.Y.Z (inherited);
- Upgrade library A to version B (application dependency);
- Apply compensating control + waiver until date (exception);
- Deadline (SLA);
- Evidence links: SBOM/provenance, scan record, policy decision.
Propagate Patches at Scale with Rebuilds and Rolling Updates
Finally, after standardizing, listing, scanning, and classifying we have approached the last stage of comprehensive vulnerability management — the actual remediation.
The goal is to turn the remediation plan into a reliable production motion: rebuild the right artifacts, roll them out safely, and prove the issues are solved successfully. That’s patch propagation at scale.
The mistake many companies make is patching per-service. But at Kubernetes scale, patching is a supply-chain workflow. Base image updates should automatically trigger the rebuild of dependent images, which should then flow into clusters through controlled rollouts.
The update cadence can follow two lanes:
- Regular updates that happen weekly or at any other interval, keeping changes minimal. They can be tied to base image updates.
- Emergency updates should happen whenever actively exploited or high-impact vulnerabilities appear and get detected in your workflows.
When a base image gets updated, every service built on that base must be automatically updated as per the following action plan:
- Rebuild application images against the new base;
- Regenerate SBOM/provenance;
- Sign the result;
- Rescan;
- Push by digest using the canary deployment strategy.
Once the rebuilt, re-scanned, and re-signed images are in the registry, roll them out across clusters. The deployment strategy must be tied to workload criticality and exposure, for example:
- Progressive delivery with canary/staged rollout for high-risk services;
- Standard rolling updates for low-risk services;
- Automatic rollback on health regression.
As a result, the running services will stay clean thanks to the regular base image updates, time-to-remediate will naturally shrink to hours/days for critical security flaws, and the teams will spend less time chasing the CVEs.
Conclusion
The CVE flow will never stop, but production containers do not need to drown in it. Resist the flood by targeting zero unmanaged risk: standardize on hardened base images, make builds deterministic and verifiable, and continuously scan across artifacts and clusters.
Patch propagation at scale closes the loop by triggering automated rebuilds from base updates and rolling signed, scanned digests through clusters safely, turning vulnerability response from a battlefield into routine maintenance.
Interested in Learning More?
Catherine Edelveis is a speaker at JCON. This article explores how to manage the constant flood of CVEs in container environments – and her JCON session shows how to turn vulnerable images into hardened, low-noise containers. If you can’t attend live, the session video will be available after the conference – it’s worth checking out!
