Managing AI Risks in the Modern Software Supply Chain

September 16, 2025
By Aaron Linskens

5 minute read time

AI-generated audio

Artificial Intelligence (AI) and Machine Learning (ML) continue to reshape software development at an unprecedented pace. Platforms like Hugging Face make millions of pre-trained models easily accessible, enabling faster innovation and powerful new applications.

Yet alongside these opportunities come significant risks — malware embedded in models, ambiguous licensing obligations, and vulnerabilities that expand the attack surface of modern software.

As AI adoption accelerates, organizations must recognize this new frontier of risk. Responsible governance and proactive security practices are essential to safeguard software supply chains without slowing innovation.

From Open Source to AI: A Familiar Pattern

The rapid adoption of AI mirrors the earlier rise of open source software (OSS). Today, roughly 90% of a typical application is made up of OSS. This shift delivered speed, sophistication, and cost savings, but also introduced major security, legal, and ethical challenges.

AI follows the same trajectory. Hugging Face alone hosts over 1.4 million models, readily available to developers. The explosion of AI adoption is creating what feels like a "gold rush." But, as history shows, speed without safety creates risks.

The record 40,000 CVEs ( common vulnerabilities and exposures) published in 2024 — a 40% jump over the previous year — illustrate what happens when technology outpaces governance. Without proactive risk management, the same scenario is inevitable for AI models.

The Expanding Risk Landscape

AI and ML models bring risks across multiple dimensions:

Security Risks: Malicious actors already embed malware in models. Just downloading or running a compromised model can trigger exploits, such as data exfiltration or corruption.
Legal and Licensing Risks: Many models carry obligations tied to their training data. Ignoring them can lead to compliance violations or lawsuits.
Quality and Ethical Risks: Outputs may be biased, unsafe, or simply wrong, undermining trust and reliability.
Intellectual Property Risks: If a model is trained on questionable data, ownership of its outputs becomes murky.

Perhaps most alarming are the technical risks posed by file formats like Python pickle files, commonly used to serialize ML models. During "unpickling," arbitrary code execution can be triggered. In 2025, Sonatype researchers uncovered multiple malicious models leveraging pickle-based exploits, proof that attackers are actively innovating in this space.

Why Visibility and Governance Are Essential

Like OSS components, AI models must be managed across the entire software development life cycle (SDLC).

Organizations need answers to questions such as:

Which models are being used?
Where are they deployed?
What risks do they introduce?

From source control to production deployment, unchecked use of AI can expose critical systems. Developers downloading models directly from public repositories bypass governance, leaving organizations blind to what is being pulled in.

To mitigate these risks, enterprises must establish policy-based governance frameworks, balancing flexibility with security. For sensitive applications, stricter policies may be warranted. For less critical workloads, leniency may be possible. But in all cases, visibility is non-negotiable.

Proactive Protection: From Policy to Prevention

Managing AI model risks requires both organizational policies and technical controls:

Model Approval Processes: Define how models are requested, approved, and consumed. Limit access to only trusted and vetted models.
Sonatype Firewall: Sonatype Firewall inspects models at the point of ingestion, blocking those with malware or policy violations before they enter the SDLC.
Continuous Monitoring: Risks evolve. Models considered safe today may later be flagged for vulnerabilities or malware. Life cycle monitoring ensures risks are caught early.
Predictive Malware Identification: Sonatype analyzes every newly published model to determine if it is malicious or suspicious, enabling fast-track review by security researchers.

Together, these measures empower organizations to move fast, without leaving the door wide open for attackers.

The Stakes Are Rising

Malware in AI models is not theoretical. It's already happening. From the rise of malicious Hugging Face models to "protestware" that sneaks into supply chains as activism, attackers are putting real effort into infiltrating development pipelines.

Additionally, the growth of "vibe coding" (AI-assisted coding) is accelerating the introduction of unintentional vulnerabilities. The combination of deliberate attacks and accidental flaws makes today's software supply chains more exposed than ever.

Ignoring these risks is no longer an option. To keep pace with rapid AI adoption, organizations must strengthen governance and enforce protections that secure model usage across the SDLC.

Building a Secure Future With AI

The growth of AI promises enormous upside, but only if organizations adopt it responsibly.

That means:

Gaining visibility into model usage.
Establishing strong governance policies.
Using automated defenses like repository firewalls.
Continuously monitoring for evolving threats.

Sonatype's solutions extend across the SDLC to deliver visibility, enforcement, and protection, helping teams adopt AI safely while maintaining speed and innovation.

The bottom line: AI and ML are already part of the modern development toolkit. The question is not whether to adopt them, but how to secure them. With the right governance and defenses, organizations can unlock the benefits of AI while keeping their software supply chains resilient.

Want to explore these risks and solutions in more detail? Watch the full recording of our webinar to gain deeper insights into securing AI in your software supply chain.

Written by Aaron Linskens

Aaron is a technical writer at Sonatype. He works at a crossroads of technical writing, developer advocacy, and information design. He aims to get developers and non-technical collaborators to work better together in solving problems and building software.