Pods: Hierarchical Configuration System

Pods

Pods are Doh's unified configuration system. Configuration uses the same hierarchical Resolution Order algorithm that governs dependency loading and pattern inheritance, ensuring consistent behavior across all systems.

The problems of application configuration—environment-specific overrides, type safety, inheritance patterns, and cross-component access—have been abstracted into a unified system that follows the same composition principles as other Doh components.

The configuration system provides:

Hierarchical file structures with inheritance patterns
Type safety through MOC validation and SeeIf assertions
Universal integration across server, browser, CLI, and infrastructure components
Namespace separation for component-specific settings
Automated compilation and caching with fingerprinting
Host-level configuration orchestration
CLI management tools for configuration lifecycle
Real-time monitoring and dashboard integration

Configuration Architecture

Pod File Structure

The configuration system organizes settings through a file structure that separates concerns based on their lifecycle and purpose:

boot.pod.yaml: Application-wide settings that define module loading and core configuration. These are version-controlled, shared settings that establish the application's baseline behavior.
pod.yaml: Instance-specific override settings for developer and deployment customization. These are typically git-ignored, enabling local configuration without polluting the shared codebase.
Module Pods: Component-specific configuration encoded through Doh.Pod() declarations, compiled into manifest data.
Package Pods: Structural configuration defined in .doh.yaml files or Doh.Package() definitions, encoding organizational settings into package metadata.

Hierarchical Composition

The configuration system uses the same Resolution Order algorithm that governs dependency loading and pattern inheritance, ensuring consistent behavior across all systems:

inherits:
  - /path/to/base.pod.yaml
  - https://example.com/shared.pod.yaml
  - /pods/custom-config.pod.yaml

This allows for configuration composition where settings cascade from base configurations to more specific ones. The inheritance chain is resolved using the build_pod function, which processes all inherited pods recursively.

Security by Design: Namespace Separation

Pod configuration embodies a deliberate security principle—separation between server and client namespaces to prevent accidental leakage of sensitive configuration to browsers:

Doh.pod: The complete pod configuration, including server-only settings like database credentials and API keys, available only in Node.js environments.
Doh.browser_pod: A curated subset of pod settings that are safe to expose to browser clients. The system automatically determines what can safely cross the client-server boundary.

This separation happens automatically—you configure once, and the system ensures sensitive data stays on the server while making appropriate settings available to client code.

Pod Integration Throughout Doh

Pods are integrated throughout the entire Doh ecosystem and automatically influence behavior across all systems:

Pre-Compilation Phase

Pods are compiled and cached before even the Auto-Packager runs, ensuring configuration is available for all subsequent operations:

// During server application startup (earliest phase)
await Doh.load_host_pod(args_pod, force_compile);

// During browser application startup
await Doh.load_browser_pod(args_pod);

Auto-Packager Integration

The Auto-Packager automatically discovers and processes pod configurations from multiple sources:

Scans .doh.yaml files in all directories for package pod definitions
Parses Doh.Pod() calls in JavaScript files using AST analysis
Processes package definitions that include pod sections
Resolves inheritance chains automatically
Generates manifests:
- /.doh/manifests/pod_manifest.json: Module pod settings from Doh.Pod() calls
- /doh_js/manifests/browser_pod.json: Browser-safe pod settings
- /.doh/compiled.pod.yaml: Full compiled pod cache

Express Router Integration

The Express router automatically uses pod settings for complete server configuration:

# All these settings come from Doh.pod.express_config
express_config:
  port: 3000                    # HTTP port binding
  ssl_port: 443                 # HTTPS port binding
  hostname: 'localhost'         # Server hostname
  ssl_info:                     # SSL/TLS certificates
    cert: '/path/to/cert.pem'
    key: '/path/to/key.pem'
  cors_hosts: []                # CORS allowed origins
  rate_limit:                   # Rate limiting settings
    windowMs: 900000
    max: 100
  host_forwards:                # Host-based routing
    'api.example.com': 'https://backend.local'
  body_size_limit: '50mb'       # Request body size limit
  cache_max_age: 86400          # Static file cache headers
  strict_hostnames: false       # Hostname validation
  log_routes: false             # Route logging

Module System Integration

Modules automatically receive pod-based configuration and support conditional loading:

Doh.Module('feature_module', [
  // Conditional loading based on pod configuration
  'Doh.pod.features.advanced?? advanced_features',
  'Doh.pod.debug_mode?? debug_tools',
  'browser&&Doh.pod.ui.enabled?? ui_components'
], function() {
  // Access configuration directly
  const config = Doh.pod.feature_module;
  const isDebug = Doh.pod.debug_mode;
  const serverConfig = Doh.pod.express_config;
});

Package Manager Integration

Dohball deployment and hosting is entirely pod-driven:

# In pod.yaml
dohball_deployment:
  expose_packages: '*'          # Which packages to expose ('*' or array)
  compile_manifest: true        # Generate hosting manifest
  rebake: false                 # Force rebuild all dohballs
  worker_pool_size: 4           # Parallel build workers
  ignore_packages: []           # Packages to exclude
  ignore_paths: ['temp/', 'cache/']  # Paths to exclude
  expose_doh_js: false          # Whether to expose core Doh files

Browser Loading Integration

Browser pods are automatically compiled and served:

// Automatically generated and included in browser loads
Doh.browser_pod = {
  // Only browser-safe configuration
  api_endpoints: { ... },
  ui_settings: { ... },
  feature_flags: { ... }
  // Server-only settings automatically excluded
};

ESBuild Integration

Build processes use pod configuration:

Doh.Pod('esbuild_doh_plugin', {
  esbuild_doh_plugin: {
    deduplicate_packages: true,
    replace_native_modules: true,
    external_modules: ['fs', 'path'],
    log_dedup_summary: false
  }
});

SSL Certificate Management

Greenlock/Let's Encrypt configuration:

letsencrypt:
  staging: false              # Use production ACME server
  force_renewal: false        # Force certificate renewal
  webroot: '/tmp/acme-challenge'
  email: 'admin@example.com'

CLI Tool Configuration

CLI behavior is pod-driven:

always_confirm_changes: true    # Require confirmation for destructive operations
always_esbuild: false          # Auto-run esbuild after changes
cleanup_orphaned_packages: true # Remove unused packages
packager_ignore:               # Directories to skip during scanning
  - 'temp/'
  - 'cache/'
  - 'logs/'

Type Safety with MOC

The Pod system uses Melded Object Composition (MOC) to provide type safety for configuration values. The moc property in a pod defines type validation for its properties using a shallow-only approach:

moc:
  apiUrl: 'IsString'
  timeout: 'IsNumber'
  retries: 'IsInt'
  database: 'IsObject'
  settings: 'IsObject'
  # Note: Unlike Pattern MOC, nested structures like this are not supported:
  # settings: 
  #   debug: 'IsBoolean'
  #   logLevel: 'IsString'

This ensures that configuration values conform to expected types and can be validated at runtime. Pod MOC validation uses a shallow approach to support proper layering of configuration files.

Pod System Components

Doh.pod

The global object containing all pod settings merged from boot.pod.yaml, pod.yaml, and module pods. It is the primary way to access configuration values in server-side code:

const port = Doh.pod.express_config.port;
const apiKey = Doh.pod.api_keys.service_name;
const features = Doh.pod.my_module.features;

Doh.browser_pod

A subset of Doh.pod that contains only browser-safe configuration values. This is made available to browser clients and excludes sensitive server-only configuration:

// In Node.js, both are available
console.log(Doh.pod.private_setting);      // Available
console.log(Doh.browser_pod.api_endpoint); // Available

// In browser, only browser_pod settings are available
console.log(Doh.pod.api_endpoint);         // Available in browser
console.log(Doh.pod.private_setting);      // Undefined in browser

Doh.Pod()

A function to define module-specific pod configuration:

Doh.Pod('my_module', {
  moc: {
    setting1: 'IsString',
    setting2: 'IsNumber'
  },
  setting1: 'default value',
  setting2: 100,
  
  // Browser-safe settings
  browser_pod: {
    publicUrl: '/api/module'
  }
});

These module pods are merged into the main pod configuration and follow the same inheritance rules.

The Boot Process

When Doh starts, it processes pod files in a specific order:

Loads core defaults from /doh_js/default.pod.yaml
Loads module-specific pods from Doh.Pod() definitions discovered by Auto-Packager
Loads package-specific pods from .doh.yaml files
Loads application settings from /boot.pod.yaml
Loads local environment settings from /pod.yaml
Executes any [[Code:x:y]]: snippets in pod definitions in a sandboxed, but pod aware Global scope.
Applies MOC validation and type checking
Generates fingerprint for cache invalidation
Separates browser-safe subset
Compiles everything into unified Doh.pod and Doh.browser_pod objects (Doh.browser_pod is also mimic'ed to Doh.pod.browser_pod so you can set it in pod files at the root level as browser_pod:)

This process ensures that local settings override application settings, which override defaults.

host_load Configuration

The host_load property in pods defines which packages should be automatically loaded when the application starts:

host_load:
  - express_router      # Web server
  - user_management     # Authentication
  - analytics          # Tracking
  - dohball_host       # Package hosting
  - storage_manager    # File management

This provides a declarative way to define your application's entry points without needing to manually load modules in code.

dohball_host Configuration

The dohball_host property defines which remote Doh instances should be used as sources for installing and upgrading remotely hosted Dohballs. This enables HTTP-based distribution of Dohballs without requiring Git. Note that this is ONLY for consuming code from external hosts - local code in your project is handled automatically by the Auto-Packager without any installation.

dohball_host:
  - https://backup-host.com
  - https://your-team-host.com
  - http://localhost:3001

How dohball_host Works

Remote Installation Only: doh install and doh upgrade commands ONLY work with remote hosts configured here
Local Code is Automatic: Code in your own project doesn't need installation - the Auto-Packager handles it automatically
Direct HTTP Installation: When you run doh install package_name, Doh searches through the configured hosts in order
Automatic Downloads: The last host that provides the requested package serves the .tar.gz file directly via HTTP
Version Comparison: doh upgrade compares locally installed versions with remote host versions to determine updates
Fallback Chain: Multiple hosts can be configured for redundancy - if the last host is unavailable, the next is tried

Hosting Dohballs

To serve Dohballs via HTTP/S, configure a Doh instance as a host:

# In the hosting instance's pod.yaml
dohball_deployment:
  compile_manifest: true
host_load:
  - dohball_host  # Enable the dohball_host module

# The hosting instance will serve baked Dohballs at:
# https://your-host.com/dohball/package_name

Distribution Methods

Dohballs support two complementary distribution approaches:

HTTP-Based (Direct): Using dohball_host for real-time distribution without Git
Git-Based (Traditional): Committing baked artifacts to Git repositories for transport

Both methods can be used simultaneously, and consumers can mix and match based on their needs.

Pod Caching and Manifests

The Pod system uses several manifest files to optimize performance:

/.doh/manifests/pod_manifest.json: Contains module pod settings compiled from Doh.Pod() calls
/doh_js/manifests/browser_pod.json: Contains browser-safe pod settings
/.doh/compiled.pod.yaml: A cache of the full compiled pod

These files are automatically managed by the Doh runtime and can be cleared with CLI commands when needed.

Advanced Features

Environment-Specific Configuration

Use pod inheritance for environment-specific settings:

# base.pod.yaml
express_config:
  port: 3000
  debug_mode: false

# development.pod.yaml  
inherits:
  - base.pod.yaml
express_config:
  debug_mode: true
  
# production.pod.yaml
inherits:
  - base.pod.yaml
express_config:
  port: 80
  ssl_port: 443
  strict_hostnames: true

Conditional Loading

Pods support conditional loading based on configuration:

Doh.Module('feature_loader_module', [
  'Doh.pod.features.advanced?? advanced_features',
  'Doh.pod.debug_mode?? debug_tools',
  'browser&&Doh.pod.ui.enabled?? ui_components'
], function() {
  // Module implementation
});

Fingerprinting and Cache Invalidation

Pods are automatically fingerprinted for cache invalidation:

// Automatically generated during compilation
pod.fingerprint = Doh.NewUUID(pod);
if (pod.browser_pod) {
  pod.browser_pod.fingerprint = pod.fingerprint;
}

Shallow MOC Validation

Pod MOC validation uses a shallow-only approach (unlike Pattern MOC) to support proper layering of configuration files. Each property's MOC definition will still meld recursively, but you cannot nest deep MOC structures with {} in your outer layer:

Doh.Pod('config_with_validation', {
  moc: {
    // Shallow MOC definitions only
    database_host: 'IsString',
    database_port: 'IsNumber',
    ssl_enabled: 'IsBoolean',
    features: 'IsArray',
    cache_ttl: 'IsNumber',
    database: 'IsObject',
    cache_settings: 'IsObject'
    // This would NOT work in pods (but works in Patterns):
    // database: {
    //   host: 'IsString',  // Deep nesting not supported
    //   port: 'IsNumber'
    // }
  },
  database_host: 'localhost',
  database_port: 5432,
  ssl_enabled: false,
  features: ['analytics', 'caching'],
  cache_ttl: 3600
});

This limitation exists because pods need to support stacking files as setting layers - when multiple pod files inherit and override each other, the MOC definitions must be able to merge cleanly without deep structural conflicts.

MOC Evolution in Static Space

We're exploring ways to expand the MOC standard into the static space of the pod engine, potentially allowing more validation patterns while maintaining the layered configuration benefits.

Nested Object MOC Workaround

For complex nested configurations, you can expose MOC directly in nested objects:

Doh.Pod('complex_config', {
  moc: {
    database: 'IsObject',
    cache_settings: 'IsObject'
  },
  database: {
    moc: {
      host: 'IsString',
      port: 'IsNumber',
      ssl: 'IsBoolean'
    },
    host: 'localhost',
    port: 5432,
    ssl: false
  },
  cache_settings: {
    moc: {
      ttl: 'IsNumber',
      max_size: 'IsNumber'
    },
    ttl: 3600,
    max_size: 1000
  }
});

This approach allows any layer in a nested pod object to define its own MOC, which will be melded with any other MOCs defined for the same key in other files, since pods are natively deep melded to represent a "static" settings perspective.

CLI Commands for Pod Management

Doh provides a unified CLI system for managing pod configurations with array detection and consistent syntax:

Unified Pod Commands

View and Set Pod Values

# Show current value and inheritance chain
doh pod express_config.port
doh bootpod host_load

# Set values
doh pod express_config.port 3000
doh bootpod debug_mode true

# Add compile-time removal directives
doh pod ~~express_config.debug
doh bootpod ~~host_load

Array Management

Arrays are detected via MOC metadata or existing values. The system provides consistent syntax:

# Add to arrays
doh pod inherits /path/to/pod.yaml
doh pod dohball_host https://host.com
doh bootpod host_load package1

# Mark for removal from arrays
doh pod inherits ~~path/to/pod.yaml
doh pod dohball_host ~~https://host.com
doh bootpod host_load ~~package1

Backward Compatibility Commands

The following commands are maintained for backward compatibility but route through the unified system:

Manage Inheritance (Legacy)

# Add inheritance (routes to: doh pod inherits /path/to/pod.yaml)
doh inherits /path/to/pod.yaml

# Mark inheritance for removal (routes to: doh pod inherits ~~path/to/pod.yaml)
doh inherits ~~/path/to/pod.yaml

Manage Host Load (Legacy)

# Add packages to host_load (routes to: doh bootpod host_load package1 package2)
doh host_load package1 package2

# Mark packages for removal from host_load (routes to: doh bootpod host_load ~~package1 ~~package2)
doh host_load ~~package1 ~~package2

Manage Dohball Hosts (Legacy)

# Add host (routes to: doh pod dohball_host https://host.com)
doh dohball_host https://host.com

# Mark host for removal (routes to: doh pod dohball_host ~~https://host.com)
doh dohball_host ~~https://host.com

Cache Management

# Clear pod cache
doh clear-pod

# Clear all caches
doh clear-all

Syntax Guide

The unified pod system uses consistent syntax:

~~value: Mark value for removal (both object keys and array items by name)
~~setting: Add compile-time removal directive (e.g., ~~debug_mode: true)
Arrays: Auto-detected via MOC metadata or existing values
Cleanup: Empty arrays are automatically removed from files
Files: doh pod edits /pod.yaml, doh bootpod edits /boot.pod.yaml

Pod Dashboard

The pod_dashboard module (if included in your application) provides a web interface for managing pod settings. It allows:

Viewing the full pod hierarchy
Editing pod values
Visualizing inheritance relationships
Testing MOC validation rules

Access it at /admin/pods in your application (with proper authentication).

Integration Points

Pods are integrated throughout the entire Doh ecosystem:

Auto-Packager: Configuration for scanning, ignoring files, build options
Express Router: Complete web server configuration
Module System: Conditional loading, feature flags, settings access
Package Manager: Dohball hosting, deployment, distribution settings
File System: Storage configuration, mount points, backup settings
Browser Environment: UI configuration, API endpoints, feature flags
CLI Tools: Operation settings, confirmation modes, output formatting
Development Tools: Debug modes, logging levels, development servers
Security Systems: SSL configuration, authentication settings, permissions
Performance: Caching, optimization, resource limits

Integration with MOC System

Pod validation leverages Doh's MOC system to support configuration layering:

moc:
  # Basic Types
  string_value: 'IsString'
  number_value: 'IsNumber'
  
  # Composite Types (shallow only)
  config_enabled: 'IsBoolean'
  config_level: 'IsInt'
  config_name: 'IsStringAndHasValue'
    
  # Advanced Validation
  domains: ['IsArray', 'NotNull']
  timeout: ['IsNumber', 'NotNull']
  database: 'IsObject'
  
  # Note: Deep nesting like this is not supported in pods:
  # database:
  #   url: 'IsKeySafe'
  #   maxConnections: 'IsIntOrFalse'

# For nested validation, define MOC in the nested objects:
database:
  moc:
    url: 'IsKeySafe'
    maxConnections: 'IsIntOrFalse'
  url: 'postgresql://localhost:5432/mydb'
  maxConnections: 10

This validation ensures configuration correctness and helps prevent runtime errors.

Pod Security Best Practices

Separate Sensitive Data: Keep API keys, database credentials, and other sensitive information in separate pod files that are ignored in version control.
Use Browser_Pod Carefully: Only expose data to the browser that is truly necessary for client-side functionality.
Validate with MOC: Always define MOC validation rules for your configurations to catch errors early.
Structure Inheritance Wisely: Create logical layers of pod files:
- Core application settings (boot.pod.yaml)
- Environment-specific settings (prod.pod.yaml, dev.pod.yaml)
- Local overrides (pod.yaml - not version controlled)
Use Path References: For file paths in pod settings, use relative paths with DohPath() in code to ensure cross-platform compatibility.
Leverage Conditional Loading: Use pod-based conditions to load features only when needed.
Cache Appropriately: Pods compile once per application start - design for this pattern.
Use Fingerprinting: Leverage automatic fingerprinting for cache invalidation in distributed systems.

Example Pod Structure

Application Settings (boot.pod.yaml)

inherits:
  - /pods/app-defaults.pod.yaml

host_load:
  - express_router
  - user_routes
  - api_service
  - dashboard

app_version: '1.2.0'
app_name: 'My Doh Application'

express_config:
  port: 3000
  
browser_pod:
  api_base: '/api/v1'
  app_name: 'My Doh Application'

Local Settings (pod.yaml)

inherits:
  - /pods/localhost.pod.yaml
  - /pods/dev-secrets.pod.yaml

express_config:
  port: 4000  # Override the default port
  
debug_mode: true

dohball_deployment:
  expose_packages: '*'
  compile_manifest: true

Module Configuration (via Doh.Pod())

Doh.Pod('user_module', {
  moc: {
    user_settings: {
      max_sessions: 'IsInt',
      default_role: 'IsString'
    }
  },
  user_settings: {
    max_sessions: 5,
    default_role: 'user'
  },
  browser_pod: {
    login_url: '/auth/login'
  }
});

The Pod system ensures that configuration is consistent, type-safe, and available throughout your entire Doh application, from server-side modules to browser-side components, CLI tools, and infrastructure management.