Secrets Management

Secrets Management

Here is a question that should make you uncomfortable: who in your multi-agent system can read your production database password right now? If the answer is “I don’t know” or “all of them,” you have a governance problem wearing an infrastructure costume.

Obsidian ’s secrets management is built on a principle that the Constitution makes non-negotiable: safety through boundaries, not trust. Zero trust for secrets. Per-agent permissions. Complete audit trails. No secrets stored in configuration files. Ever.

Architecture

◈ Obsidian Secrets Layer 

┌─────────────┐    ┌─────────────────────────────────────┐
│   Agents    │───▶│          Secrets Manager             │
└─────────────┘    │  ┌──────────────────────────────┐   │
                   │  │       Access Control          │   │
┌─────────────┐    │  │  Per-agent permissions        │   │
│   Config    │───▶│  │  Path-based policies          │   │
│  Resolver   │    │  │  Audit logging                │   │
└─────────────┘    │  └──────────────────────────────┘   │
                   │               │                      │
                   │  ┌──────────────────────────────┐   │
                   │  │         Cache Layer           │   │
                   │  │  Encrypted in-memory cache    │   │
                   │  │  TTL-based expiration         │   │
                   │  │  Automatic refresh            │   │
                   │  └──────────────────────────────┘   │
                   │               │                      │
                   │  ┌──────────────────────────────┐   │
                   │  │      Router / Resolver        │   │
                   │  │  Pattern matching             │   │
                   │  │  Provider selection           │   │
                   │  │  Fallback handling            │   │
                   │  └──────────────────────────────┘   │
                   └─────────────────────────────────────┘
                                   │
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│Infisical │ │  Vault   │ │ AWS SSM  │ │  Azure   │ │   GCP    │
│ Provider │ │ Provider │ │ Provider │ │ Key Vault│ │ Secrets  │
└──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘

The architecture is layered with deliberate intent. Agent requests pass through access control first — not as middleware, but as the gate. Then through an encrypted cache (because hitting the provider on every request is both slow and expensive). Then through a router that selects the right provider based on pattern matching. Only then does the request reach the actual secrets backend.

This is Constitution Principle 6 in action: no vendor lock-in, no single point of failure. You can run Infisical for development, HashiCorp Vault for staging, and AWS Secrets Manager for production — simultaneously. The router handles the mapping. The agents never know the difference.

Five Design Principles

The secrets system is built on five non-negotiable principles:

  1. Provider Agnostic — support multiple secret providers simultaneously, because any single provider is a single point of failure
  2. Secure by Default — encryption at rest and in transit, because unencrypted secrets are not secrets
  3. Fine-grained Access — per-agent secret permissions, because “everyone can read everything” is not a security model
  4. Audit Trail — complete logging of every secret access, because accountability requires visibility
  5. Zero Trust — no secrets stored in configuration files, because trust is not a security mechanism

Built-in Providers

Obsidian ships with providers for every major secrets platform. Each provider implements the same interface — get, set, delete, list, exists, rotate — which means switching providers is a configuration change, not a rewrite.

Infisical

Enterprise-grade open-source secrets management. Supports Universal Auth (recommended for production), service tokens (legacy), and Kubernetes machine identity. Self-hosted or cloud. Obsidian’s recommended default for teams that want to own their secrets infrastructure.

HashiCorp Vault

The industry standard. Supports token auth, AppRole, Kubernetes, AWS IAM, and LDAP authentication. Works with KV v1, KV v2, Transit, and PKI secrets engines. If your organization already runs Vault, Obsidian integrates natively.

AWS Secrets Manager and SSM Parameter Store

Native AWS secrets management for teams running on AWS. Supports the full credential chain — environment variables, IAM roles, profiles, and assumed roles. SSM Parameter Store provides a lighter-weight alternative with path-prefix organization.

Azure Key Vault

Service Principal, Managed Identity, or Azure CLI authentication. Full integration with Azure’s secrets lifecycle including soft-delete and purge protection.

GCP Secret Manager

Service account and application default credentials. Regional and global secret management with automatic replication policies.

Environment Variables

The simplest provider — reads from the process environment. Useful for development, CI/CD pipelines, and twelve-factor app patterns. Not recommended for production secrets management, but Obsidian does not pretend development environments need enterprise secret infrastructure.

Secret Routing

The router is what makes multi-provider secrets practical. You define rules that match secret paths to providers:

secrets:
  routing:
    rules:
      - pattern: "api-keys/*"
        provider: "infisical"
      - pattern: "infrastructure/*"
        provider: "vault"
      - pattern: "cloud/aws/*"
        provider: "aws-secrets"
      - pattern: "cloud/azure/*"
        provider: "azure"
      - pattern: "cloud/gcp/*"
        provider: "gcp"
    fallback: "infisical"

When an agent requests api-keys/openai, the router matches the pattern and routes to Infisical. When it requests infrastructure/database-url, the router selects Vault. The fallback catches anything that does not match an explicit rule.

This is not just convenience. It is operational resilience. If your Vault instance goes down, your AWS secrets still work. If Infisical has an outage, your infrastructure secrets — stored in Vault — are unaffected. The providers are independent. The failures are isolated.

Encrypted Caching

Secrets are cached in-memory with encryption. Not because you do not trust your own process, but because defense in depth means every layer assumes every other layer has been compromised.

The cache uses TTL-based expiration with configurable refresh ahead — secrets are refreshed before they expire, so agents never hit a cold cache during critical operations. Cache encryption uses AES-256-GCM with a derived key that rotates with the process lifecycle.

secrets:
  cache:
    enabled: true
    default_ttl: 300       # 5 minutes
    max_ttl: 3600          # 1 hour ceiling
    encryption:
      algorithm: "aes-256-gcm"
      key_derivation: "hkdf-sha256"
    refresh_ahead: true
    refresh_threshold: 0.8  # Refresh at 80% of TTL

Access Control

Every Agent declares which secrets it is allowed to access, and those declarations are enforced by the secrets manager — not by the agent itself. This is the difference between a suggestion and a boundary.

agent:
  id: "worker-alpha"
  secrets:
    allowed_paths:
      - "api-keys/openai"
      - "api-keys/anthropic"
      - "database/read-*"
    denied_paths:
      - "infrastructure/*"
      - "admin/*"
    max_access_rate: 10  # per minute

Worker Alpha can read the OpenAI and Anthropic API keys. It can read any database secret starting with read-. It cannot touch infrastructure secrets or admin credentials. If it tries, the request is denied and logged. The Inquisitor will have questions.

Audit Logging

Every secret access generates an audit log entry. Who accessed what, when, from where, and whether it succeeded. This is not optional. This is Constitution Principle 2: if it isn’t observable, it doesn’t exist.

{
  "timestamp": "2024-01-15T10:30:00Z",
  "event": "secret.access",
  "agent": "worker-alpha",
  "path": "api-keys/openai",
  "provider": "infisical",
  "action": "read",
  "result": "success",
  "cached": true,
  "duration_ms": 2
}

Failed access attempts get logged with the same fidelity. Denied access attempts get elevated severity. The audit log is the system’s memory of who touched what — and in a zero-trust architecture, memory is accountability.

Secret Rotation

Secrets rot. API keys get compromised. Passwords age. Obsidian supports both manual rotation via obs secrets rotate and automatic rotation policies defined per-secret:

secrets:
  rotation:
    policies:
      api-keys:
        interval_days: 90
        pre_rotation_hook: "validate-new-key"
        post_rotation_hook: "notify-dependent-agents"
        notify_before_days: 7
      database:
        interval_days: 30
        auto_rotate: true

Rotation is not just replacing a value. It is a pipeline: validate the new secret works, swap it in, notify dependent agents, verify nothing broke, and log the entire sequence. Because a rotation that breaks production is worse than an old secret.

The Bootstrap Problem

How do you authenticate to your secrets provider without having a secret to authenticate with? This is the bootstrap problem, and every secrets system has to solve it.

Obsidian’s answer is pragmatic: the bootstrap secret — and only the bootstrap secret — comes from the environment. One environment variable to authenticate to the primary secrets provider. Everything else is retrieved from the provider. This minimizes the attack surface to a single credential while keeping the rest of the secrets ecosystem fully managed.

secrets:
  bootstrap:
    provider: "infisical"
    auth:
      type: "universal-auth"
      clientId: "${env:INFISICAL_CLIENT_ID}"
      clientSecret: "${env:INFISICAL_CLIENT_SECRET}"

Two environment variables. One provider connection. Everything else flows from there. The bootstrap is the single point of trust in a zero-trust system — and it is protected, rotated, and audited with the same rigor as everything else.