AI Supply Chain Security: Risks in Models, Libraries, Plugins and Data

An AI application can be compromised even when its own source code is clean. Modern AI systems depend on a chain of external components: foundation models, fine-tuned adapters, Python packages, container images, model repositories, datasets, embedding models, plugins, APIs, vector databases and cloud services. Each dependency introduces trust that the application owner may not fully control.

That makes AI supply chain security broader than ordinary dependency management. Traditional software supply chain controls still matter, but AI adds components whose behavior can change without looking like normal executable code. A model can contain a backdoor. A dataset can be poisoned. A plugin can gain excessive access. A model file can be packaged in a format that executes code when loaded. A hosted model or API can also change behind a stable endpoint.

OWASP’s current LLM03:2025 Supply Chain guidance explicitly includes traditional third-party packages, pre-trained models, LoRA adapters, weak model provenance and external data among the risks. NIST’s SP 800-218A extends the Secure Software Development Framework with practices specific to AI model development throughout the lifecycle.

The practical goal is not to distrust every third party. It is to make every important dependency identifiable, reviewable, constrained and replaceable.

What is the AI supply chain?

The AI supply chain is the set of people, systems and artifacts that can influence an AI application’s behavior before and during operation.

A typical generative AI application may rely on:

• a foundation model from an external provider or model hub;
• a fine-tuned model, LoRA adapter or other model modification;
• training, fine-tuning, evaluation or retrieval datasets;
• tokenizers, embedding models and rerankers;
• machine learning frameworks and ordinary software libraries;
• container images, GPU runtimes and operating system packages;
• model-serving platforms and inference APIs;
• plugins, tools, MCP servers or other agent integrations;
• vector databases and external knowledge sources;
• CI/CD pipelines, registries, secrets and cloud infrastructure.

Any one of these can alter what the system executes, retrieves, learns from, reveals or is allowed to do.

This is closely related to software supply chain attacks, but AI creates additional integrity questions. With normal software, teams often ask whether a binary came from reviewed source. With AI, they must also ask where a model came from, what data influenced it, whether an adapter changed its behavior, whether a tool description can be manipulated, and whether the deployed model is still the same artifact that was evaluated.

Why models are supply chain components

A downloaded model should not be treated as a harmless data file simply because its purpose is machine learning.

Model artifacts can be large binary objects that are difficult to inspect manually. Some serialization formats may also support code execution during deserialization. OWASP’s LLM03 guidance therefore treats third-party pre-trained models as part of the supply chain, not merely as content.

There are two different security questions to separate.

First, is the model artifact itself safe to load? This includes file integrity, serialization format, publisher identity and the possibility of malicious code or tampering.

Second, is the model behavior trustworthy for this application? A model can load safely and still contain hidden behavior, unexpected biases, unsafe capabilities or a targeted backdoor. Static malware scanning cannot answer that second question by itself.

A stronger model intake process therefore combines provenance checks with isolated loading, safety evaluation and application-specific testing.

Libraries and frameworks still matter

AI systems do not stop being software systems. They still depend on package managers, web frameworks, SDKs, HTTP clients, parsers, notebooks, container images and operating system libraries.

A vulnerable or malicious package can compromise the AI application before any model inference occurs. The consequences can be especially serious because AI development environments may hold model repository tokens, cloud credentials, proprietary datasets and access to powerful GPU infrastructure.

Lockfiles, dependency review, package integrity verification and vulnerability management remain important. For high-risk production systems, teams should also consider controlled internal mirrors, approved package sources and restrictions on install-time scripts.

The key mistake is creating a separate “AI security” process while ignoring conventional application security. AI supply chain security sits on top of secure software engineering, not instead of it.

Data is also part of the supply chain

Training and retrieval data can influence system behavior without containing executable code.

OWASP’s LLM04:2025 Data and Model Poisoning describes how manipulated pre-training, fine-tuning or embedding data can introduce vulnerabilities, backdoors or biased behavior. This means a dataset supplier, scraping pipeline or external knowledge feed can become a security boundary.

Suppose a support chatbot uses a retrieval system that regularly imports documents from a shared external source. If an attacker can modify that source, they may not need access to the model or application code. They can instead influence the information retrieved at runtime. That overlaps with indirect prompt injection when malicious instructions are embedded in external content, and with data poisoning when the attacker aims to corrupt knowledge or learned behavior.

Useful controls include source allowlists, ownership checks, ingestion logging, content review for high-impact corpora, versioned datasets, cryptographic hashes for immutable artifacts and the ability to trace retrieved content back to its origin.

Plugins, tools and agent integrations expand the chain

AI agents may connect to email, file storage, databases, browsers, code execution systems, payment services or internal APIs. Those integrations can become part of the supply chain because their descriptions, schemas, permissions and implementation influence what the agent is capable of doing.

A tool does not need to be malicious to create risk. It may simply request more privilege than the application requires. A compromised integration can be worse because the agent may treat it as trusted infrastructure.

This is why tool security needs both supplier trust and runtime authorization. Verifying who published a plugin is useful, but it does not justify giving that plugin unrestricted access.

For MCP-based systems, the same principle applies to servers and tools. See MCP Security Risks for a deeper look at tool poisoning, secret exposure and excessive permissions.

A practical trust-boundary model

An AI application can be reviewed as a sequence of trust decisions:

The table highlights an important distinction: inventory is not verification. Knowing that a system uses a certain model or package is useful, but security depends on proving that the exact artifact and configuration are the ones you intended to trust.

Pin what can change

Mutable references are convenient but dangerous in security-sensitive pipelines.

A container tag such as latest, a floating package range, a model alias or an unversioned API can point to different content over time. That makes it harder to reproduce testing and investigate incidents.

Where the ecosystem supports it, prefer immutable identifiers such as exact package versions, commit revisions, model revisions and container digests. For example:

model:
  repository: approved-org/support-model
  revision: "<reviewed-immutable-revision>"

runtime:
  image: registry.example.com/ai-service@sha256:<approved-digest>

The placeholders above are intentionally not real identifiers. The security idea is that deployment should consume the exact revision approved by the release process rather than silently resolving whatever happens to be newest.

Pinning does not prove an artifact is safe. It prevents unnoticed substitution after review and makes later analysis much more reliable.

Build an AI bill of materials

A conventional Software Bill of Materials can document packages and software components, but an AI system often needs additional inventory.

For each production AI service, record at least:

• model name, provider, version or immutable revision;
• fine-tuning adapters and their origin;
• important datasets and ingestion sources;
• embedding and reranking models;
• libraries, containers and runtime dependencies;
• external tools, plugins and MCP servers;
• hosted APIs and model endpoints;
• licenses and usage restrictions where relevant;
• hashes, signatures or provenance records when available.

This does not require inventing a new standard before starting. A simple machine-readable inventory linked to the release can already improve incident response. If a model, adapter or package is later found to be unsafe, teams can answer the practical question: where are we using it?

NIST SP 800-218 and SP 800-218A emphasize provenance and secure development practices, while NIST’s SBOM guidance explains how component inventories improve transparency in software supply chains. AI teams can extend the same idea to model and data artifacts.

Treat model hubs and marketplaces as external suppliers

Model hubs, plugin directories and package registries provide enormous value, but popularity is not proof of security.

Before importing a third-party model or tool, check the publisher identity, repository history, documentation, license, artifact format and maintenance status. Prefer verified or organization-controlled publishers where possible. Review files before loading them into privileged environments and perform initial evaluation in an isolated environment with restricted credentials and network access.

Do not give a newly downloaded model access to production secrets simply because it came from a well-known platform. The platform may provide scanning and moderation, but the application owner still owns the risk decision.

Separate evaluation from production trust

A model that performs well on quality benchmarks has not automatically passed a supply chain review.

Performance evaluation asks whether the model produces useful outputs. Security evaluation asks different questions: whether the artifact is authentic, whether loading it is safe, whether it behaves unexpectedly under adversarial conditions, whether it leaks sensitive data, and whether it stays within the application’s intended boundaries.

For important models, maintain a repeatable acceptance test suite. If a model revision, adapter, system prompt, retrieval source or provider version changes, rerun the relevant tests before promotion.

That is especially important for hosted models where the provider controls infrastructure you cannot inspect directly. Contracts, provider assurance and versioning matter, but your own regression tests provide evidence about the behavior your application actually depends on.

Protect the environment that handles AI artifacts

AI development systems are attractive targets because they combine valuable data with powerful infrastructure.

Use least-privilege identities for model downloads, training jobs and deployment. Separate production release credentials from ordinary developer accounts. Restrict outbound network access from sensitive build or model-evaluation jobs when practical. Avoid placing long-lived cloud secrets inside notebooks or model configuration files.

CI/CD changes that alter model revisions, tool permissions, data sources or deployment images should receive the same review as application code. A one-line model reference change can materially change production behavior.

What to do when an AI dependency becomes untrusted

If a model, package, dataset or integration is suspected of compromise, start by stopping further promotion of the affected artifact. Preserve the version, hashes, logs and provenance records needed for investigation.

Then determine the blast radius. Identify which applications, environments and users consumed the component. Review what credentials and data the affected stage could access. If execution compromise is possible, rotate exposed credentials and investigate the surrounding host or pipeline rather than replacing only the model file.

For data or model poisoning, replacing the artifact may not be enough if derived assets remain. Fine-tuned checkpoints, embeddings, caches or downstream datasets may need to be regenerated from a trusted point.

Finally, restore through a known-good chain: trusted source, reviewed inputs, controlled build or training process, fresh evaluation and verified deployment artifact.

The security principle that ties it together

AI supply chain security is fundamentally about controlling inherited trust.

Every external model, dataset, package and tool saves time because someone else has already built something useful. The security cost is that your system inherits assumptions about that supplier’s identity, development process, infrastructure and future changes.

Strong teams make those assumptions visible. They inventory dependencies, pin important artifacts, verify provenance where possible, isolate unknown components, test behavior, limit permissions and keep enough records to replace a compromised dependency without guessing.

The result is not a perfectly trusted AI stack. It is a stack where trust decisions can be explained, checked and changed when new evidence appears.

Sources and further reading

• OWASP LLM03:2025 Supply Chain
• OWASP LLM04:2025 Data and Model Poisoning
• NIST SP 800-218A: Secure Software Development Practices for Generative AI and Dual-Use Foundation Models
• NIST AI 600-1: Generative Artificial Intelligence Profile
• NIST Secure Software Development Framework
• NIST Software Security in Supply Chains: Software Bill of Materials