Docs/Failure-Mode Library

AI Trust Failure-Mode Library

Reference failure modes for AI systems. Each entry describes the failure mode, affected architecture, potential impact, relevant Vallum capabilities, evidence Vallum can produce, and limitations.

Important: These entries describe reference failure modes. They are not error codes emitted by the named third-party platforms unless explicitly documented otherwise. They represent architectural risk patterns that may occur in AI systems, along with how Vallum capabilities may help address specific aspects of each failure mode.

VTF-001

Unbounded Agent Output in Regulated Deployment Pipeline

Affected Architecture

AI agents integrated into CI/CD or deployment workflows

Preconditions

An AI agent produces outputs that vary across invocations. The deployment pipeline requires verifiable, bounded outputs for compliance purposes. No structural boundary exists between the agent's output and the downstream system.

Potential Impact

Unverifiable agent behavior in production. Inability to reproduce or audit specific outputs. Compliance gaps for frameworks requiring system monitoring (e.g., SOC2 CC7.2). Difficulty demonstrating that deployed behavior matches tested behavior.

Relevant Vallum Capability

Titan Handshake can receipt the authorization decision before a deployment proceeds. Titan Verify can assemble evidence showing what was authorized, by whom, and under what policy. This creates an auditable record of the deployment decision — though it does not make the underlying model output deterministic.

Evidence Vallum Can Produce

Authorization receipts, policy evaluation records, deployment decision evidence packages

Limitations

Vallum does not make LLM outputs deterministic. It does not intercept or modify model outputs. It provides evidence of authorization decisions, not guarantees about model behavior. Output validation and schema enforcement are application-layer responsibilities.

Additional External Controls Required

Application-level output validation, model testing frameworks, CI/CD security policies, deployment approval workflows

VTF-002

Cross-Tenant Data Exposure in Multi-Tenant AI System

Affected Architecture

Multi-tenant AI applications with shared infrastructure (vector databases, model endpoints, caching layers)

Preconditions

Multiple tenants share underlying infrastructure. Application-level tenant filtering contains a bug or misconfiguration. A query from Tenant A returns data belonging to Tenant B.

Potential Impact

Data breach affecting one or more tenants. Regulatory notification obligations (GDPR Article 33, CCPA). Loss of customer trust. Potential legal liability for unauthorized data disclosure.

Relevant Vallum Capability

Titan Handshake enforces tenant-scoped authorization. Protected operations within Verified Ops are tenant-isolated — wrong-tenant authorization fails closed. Evidence packages are tenant-scoped and private evidence is never exposed through public verification endpoints.

Evidence Vallum Can Produce

Tenant-scoped authorization receipts, access denial records for cross-tenant attempts, tenant isolation boundary logs

Limitations

Vallum does not control your vector database, application memory store, or caching layer. Tenant isolation within Vallum applies to Vallum-protected operations and Verified Ops evidence. Your application must implement its own tenant isolation for data stores outside Vallum's scope. Vallum cannot prevent cross-tenant leakage in systems it does not manage.

Additional External Controls Required

Application-level tenant isolation in data stores, vector database namespace enforcement, infrastructure-level network segmentation, regular penetration testing

VTF-003

Uncontrolled Metadata Disclosure via Agent Tool Calls

Affected Architecture

AI agents with external API access (tool-use, function-calling, plugin systems)

Preconditions

An AI agent makes outbound API calls to third-party services. The agent includes operational metadata (user identifiers, internal state, business context) in request parameters or headers. No boundary exists between internal context and outbound communications.

Potential Impact

Gradual disclosure of business intelligence through aggregated metadata. Competitive intelligence leakage. Privacy violations if personal data is included in outbound requests. Potential regulatory exposure under data minimization requirements.

Relevant Vallum Capability

Vallum can receipt and audit outbound operations when they are routed through Titan Handshake. Authorization receipts document what operations were approved and under what context. However, Vallum does not inspect, filter, or modify the content of outbound API calls.

Evidence Vallum Can Produce

Authorization receipts for outbound operations, audit trail of approved external communications

Limitations

Vallum does not inspect or filter outbound API call payloads. It does not operate at the network layer to strip metadata from HTTP requests. Content filtering, data loss prevention, and outbound request sanitization are application-layer or network-layer responsibilities outside Vallum's scope.

Additional External Controls Required

Outbound request sanitization middleware, data loss prevention (DLP) tools, network-layer egress controls, API gateway policies, agent tool-call allowlists

VTF-004

Unauthorized Operation Execution via Instruction Manipulation

Affected Architecture

AI agents with tool access where permissions are defined in prompt context

Preconditions

An AI agent's permissions are defined within its prompt or instruction context. An attacker injects instructions (via user input, processed documents, or retrieved content) that cause the agent to execute operations beyond its intended scope. No external permission enforcement exists.

Potential Impact

Unauthorized data access, record modification, or workflow execution. Privilege escalation that bypasses intended access controls. Audit trails that appear legitimate (the agent 'chose' to act) making detection difficult.

Relevant Vallum Capability

Titan Handshake enforces authorization externally — outside the agent's instruction context. If an operation requires a handshake, the authorization decision is made based on the tenant's declared permissions, not the agent's runtime instructions. This means a prompt injection cannot grant permissions that were not structurally authorized.

Evidence Vallum Can Produce

Authorization denial receipts for out-of-scope operations, policy evaluation records showing which permissions were checked

Limitations

Vallum only protects operations that are routed through Titan Handshake. If an agent can execute operations without requiring a handshake (e.g., direct database access, unprotected API calls), Vallum cannot prevent unauthorized execution. Comprehensive protection requires that all sensitive operations are gated behind handshake authorization. Vallum does not prevent the prompt injection itself — it limits the damage an injected instruction can cause for handshake-protected operations.

Additional External Controls Required

Comprehensive operation gating (all sensitive operations require authorization), input sanitization, agent sandboxing, least-privilege infrastructure access, monitoring for anomalous agent behavior

VTF-005

Incomplete Audit Trail for Automated Operational Decisions

Affected Architecture

AI-assisted operational workflows (deployment approvals, configuration changes, artifact releases)

Preconditions

An AI system participates in operational decisions (e.g., approving a deployment, recommending a configuration change). The decision is executed but no structured evidence records why the decision was made, what inputs were considered, or what policies were evaluated.

Potential Impact

Inability to demonstrate compliance with audit requirements. Difficulty investigating incidents involving automated decisions. Regulatory exposure for frameworks requiring decision documentation (EU AI Act Article 14, SOC2 CC7.2). Inability to answer 'why did the system do this?' during post-incident review.

Relevant Vallum Capability

Titan Handshake receipts capture the decision context (what was requested, policies evaluated, authorization outcome) before execution. Titan Verify assembles these receipts into evidence packages. Titan Sign seals the evidence for cryptographic provenance. Together, they create a structured audit trail for operations routed through Vallum.

Evidence Vallum Can Produce

Decision receipts with full context, evidence packages with receipt chains, cryptographically sealed audit records, public verification URLs for certificate status

Limitations

Vallum only produces audit evidence for operations routed through its infrastructure. Decisions made outside Vallum (direct API calls, manual actions, operations in systems without Vallum integration) are not captured. Vallum does not automatically package every decision in your system — only those explicitly integrated with Titan Handshake. Complete audit coverage requires integrating all auditable operations with Vallum.

Additional External Controls Required

Comprehensive integration of auditable operations with Titan Handshake, application-level logging for non-Vallum operations, log aggregation and SIEM, regular audit gap assessments

Need help assessing your architecture?

Our engineering team can review your specific AI architecture and identify which failure modes apply to your system, along with where Vallum integration may help and where additional controls are needed.

Request Architecture Review