Manifests: Codebase tracking

Manifests are JSON files that Doh's Auto-Packager generates through AST analysis to track and manage your application's structure. These manifests contain metadata extracted from static code analysis, including dependency graphs, function signatures, inheritance chains, and cross-references. While these files are managed automatically, understanding their purpose and contents can help you debug issues and understand how your application works.

This guide covers the main manifests generated by the Auto-Packager, categorized by their location and purpose.

Manifest Locations

Manifests are primarily stored in two locations:

• /.doh/manifests/ (Private): Contain internal build information, caches, and configuration. They are not distributed and are excluded from being served by default.
• /doh_js/manifests/ (Public): Describe the public structure and dependencies of your application. They are often used by the framework at runtime and may be included in distributions or served.
• /dohballs/ (Hosting Root): Contains manifests related to the Dohball hosting mechanism itself. (One lone manifest)

Private Manifests (/.doh/manifests/)

1. Auto-Packager Cache (ap_cache.json)

Purpose: Stores file metadata (modification time, content hash), parsed Doh definitions (modules, packages, patterns), and import information. This cache speeds up subsequent Auto-Packager runs by only reprocessing changed files.

Location: /.doh/manifests/ap_cache.json

Usage: Internal to the Auto-Packager. Inspecting it can help debug caching issues.

2. CLI Manifest (cli_manifest.json)

Purpose: Defines available CLI commands registered via Doh.CLI() and maps them to their implementation script files.

Location: /.doh/manifests/cli_manifest.json

Example Entry:

{
  "users": {
    "poduser": {
      "file": "modules/database/manage_pod_users.js",
      "help": "[/path/to/pod.yaml] Manage users stored in pod.yaml files (Default: /pod.yaml)"
    }
  }
}

Usage: Used by the doh command-line tool to find and execute registered commands.

3. Pod Manifest (pod_manifest.json)

Purpose: Aggregates Pod configuration snippets defined across various modules using Doh.Pod().

Location: /.doh/manifests/pod_manifest.json

Example Entry:

{
  "express_router": {
    "express_config": {
      "port": 3000,
      "JWT_SECRET": "your-secret-key",
      // ... other express settings
    }
  }
}

Usage: Used internally by Doh to assemble the final, merged pod configuration during startup.

4. Node ESM Manifest (node_esm_manifest.json)

Purpose: Import map intended for Node.js environments. It maps imported package names (e.g., 'axios') to their resolved relative file paths within the node_modules directory. Note: This manifest is currently generated but not actively used by the core framework.

Location: /.doh/manifests/node_esm_manifest.json

Example Entry:

{
  "axios": "node_modules/axios/index.js",
  "react": "node_modules/react/index.js"
}

Usage: Potentially useful for external tools or future Node.js execution contexts needing direct path resolution. Currently informational.

5. Autopackager Problems (autopackager_problems.json)

Purpose: Logs errors, warnings, or inconsistencies detected during the Auto-Packager run (e.g., missing dependencies, parsing errors).

Location: /.doh/manifests/autopackager_problems.json

Usage: For diagnosing issues encountered during the doh update or build process.

6. Pattern Duplicates (pattern_duplicates.json)

Purpose: Lists any patterns that have been defined with the same name in multiple different files, along with their locations.

Location: /.doh/manifests/pattern_duplicates.json

Usage: Helps identify naming conflicts or redundant pattern definitions.

7. Deprecated Features (deprecated_features.json)

Purpose: Logs usage of Doh functions, parameters, or features that are marked as deprecated, indicating where they are used in the code.

Location: /.doh/manifests/deprecated_features.json

Example Entry:

[
  [
    "The globals parameter is deprecated in Doh.Module:",
    "my_module",
    "modules/my_module/code.js:123",
    "for globals:",
    "some_global"
  ]
]

Usage: Helps identify and update code using outdated parts of the Doh API.

8. Compiled Dohball Manifest (compiled_dohball_manifest.json)

Purpose: Acts as an internal cache storing the contents of dohball_manifest.json files fetched from remote Dohball hosts configured in pod.yaml.

Location: /.doh/manifests/compiled_dohball_manifest.json

Usage: Used by the installer/upgrader to resolve remote Dohball dependencies without repeatedly fetching manifests from hosts.

Public Manifests (/doh_js/manifests/)

1. Package Manifest (package_manifest.json)

Purpose: The central registry of all discovered packages (both local and from installed Dohballs). Defines their dependencies (load), associated module parameters, source file location, and other metadata. The params captured here enable Scoped Parameter Resolution (SPR)—where module callbacks consume names that earlier modules produced without re-importing (see Modules → SPR).

Location: /doh_js/manifests/package_manifest.json

Example Entry:

"my_package": {
  "module_name": "my_package",
  "load": [
    "core_dependency",
    "import externalLib from \"external-lib\""
  ],
  "params": [ /* callback parameter specs captured by Auto-Packager (simple names, destructured entries, alias defaults) */ ],
  "file": "/modules/my_stuff/my_package.js",
  "path": "modules/my_stuff",
  "install": { /* installation requirements */ }
}

Usage: Used by the Doh loader to resolve dependencies and load modules/packages.

2. Pattern Manifest (patterns_manifest.json)

Purpose: Maps pattern names to the name of the module they are defined within.

Location: /doh_js/manifests/patterns_manifest.json

Example Entry:

{
  "MyButton": "ui_widgets",
  "UserProfileCard": "user_features"
}

Usage: Used by the framework to locate and instantiate patterns.

3. Core Patterns Manifest (core_patterns_manifest.json)

Purpose: Lists the names of all patterns provided by the core doh_js framework itself.

Location: /doh_js/manifests/core_patterns_manifest.json

Example Entry: [ "control", "html", "element", "button", ... ]

Usage: Informational, helps distinguish between application-defined and framework-provided patterns.

4. Assets Manifest (assets_manifest.json)

Purpose: Maps module names to a list of relative asset paths detected within that module (specifically from DohPath.DohSlash() calls).

Location: /doh_js/manifests/assets_manifest.json

Example Entry:

{
  "my_ui_module": [
    "/images/logo.png",
    "/styles/main.css"
  ]
}

Usage: Used by build tools or deployment scripts to identify static assets associated with specific modules.

5. Browser ESM Manifest (browser_esm_manifest.json)

Purpose: Provides an import map specifically for browser environments. It maps imported package names (e.g., 'axios') to browser-loadable URLs, typically pointing to esm.sh CDN URLs generated based on versions found in node_modules.

Location: /doh_js/manifests/browser_esm_manifest.json

Example Entry:

{
  "imports": {
    "axios": "https://esm.sh/axios@1.6.0",
    "react": "https://esm.sh/react@18.2.0"
  }
}

Usage: Used to construct <script type="importmap"> tags, enabling standard import syntax for NPM packages in browser code.

6. Dohball Manifest (dohball_manifest.json)

Purpose: If dohball_deployment.compile_manifest is true in pod.yaml, this file lists the locally baked Dohballs that this Doh instance can serve to other instances. Includes package info, version, and update timestamp.

Location: /doh_js/manifests/dohball_manifest.json

Example Entry:

{
  "my_shared_package": {
    "load": [ /* dependencies */ ],
    "path": "modules/shared",
    "packagefile": "/modules/shared/package.doh.yaml",
    "version": "1.2.3",
    "updated": "2024-09-21T12:45:32.603Z"
    /* other package metadata */
  }
}

Usage: Allows other Doh instances (configured with this instance's URL as a dohball_host) to discover and download available Dohballs.

7. Module Dependency Graph (module_dep_graph.json)

Purpose: Represents the direct dependency relationships between packages/modules as declared in their load arrays.

Location: /doh_js/manifests/module_dep_graph.json

Example Entry:

{
  "module_a": [ "core_utils", "module_b" ],
  "module_b": [ "core_utils" ]
}

Usage: For visualizing the dependency structure of the application and detecting cyclic dependencies (though the packager performs cycle checks).

Dohball Hosting Manifest (/dohballs/)

1. Orphaned Dohballs (dohballs.json)

Purpose: Tracks Dohball archives (.tar.gz files) that exist within the /dohballs/ hosting directory but no longer correspond to a package currently configured for exposure via pod.yaml (i.e., they are "orphaned").

Location: /dohballs/dohballs.json

Example Entry:

{
  "removals": [
    "/modules/old_feature",
    "/experimental/temp_package"
  ]
}

Usage: Helps identify obsolete Dohball archives that might be safe to remove from the hosting directory. It does not automatically delete files.

Best Practices

1. Do Not Edit Directly: These manifests are generated and managed by the Auto-Packager. Manual edits will be overwritten and can cause unpredictable behavior.
2. Version Control: Manifests are designed to be local and replace the normal "build" output from Doh. We still build your usual JS modules ESM/CJS/UMD/AMD/ETC in /dist, but Doh files are designed to be non-built and instead ship as source, optionally with manifests, though most rely on the Auto-packager.
3. Debugging Aid: Use these manifests, especially autopackager_problems.json, package_manifest.json, and module_dep_graph.json, to understand your application's structure, trace dependencies, and diagnose loading or build issues.