The Managed Site module turns a Doh instance into a remotely manageable agent with secure cloud anchoring, persistent connections, and a rich remote command system.
doh cloud endpoint https://deploydoh.com
# To your own account
doh cloud anchor
# Or anchor on behalf of another user (requires manage:cloud_anchoring)
doh cloud anchor-as user@example.com
curl http://localhost:3000/api/managed_site/status
# boot.pod.yaml or pod.yaml
cloud:
mcp_controllable: true
Then start your local MCP server: doh mcp. See MCP Integration.
host_load:
- managed_site
cloud:
endpoint: 'https://deploydoh.com' # Cloud manager endpoint
token_storage_path: '/.doh/cloud_auth_token' # Auth token location
cloud_anchor_path: '/.doh/cloud-anchor.json' # Persistent fingerprint storage
connection_timeout: 30000 # Connection timeout (30s)
reconnect_interval: 5000 # Reconnection interval (5s)
heartbeat_interval: 60000 # Heartbeat frequency (1m)
mcp_controllable: false # Enable MCP remote control (SECURITY: defaults to false)
reverse_proxy: # Reverse proxy configuration (optional)
enabled: true # Enable reverse proxy through cloud manager
external_port: 4443 # External port on cloud manager for this instance
file_sync: # Advanced file transfer settings
max_chunk_size: 1048576 # 1MB chunks
min_chunk_size: 65536 # 64KB minimum
max_concurrent_transfers: 3 # Concurrent transfers
transfer_timeout: 3600000 # 1 hour timeout
integrity_check: true # Enable SHA256 checksums for verification
md5_cache_enabled: true # Enable MD5 sync status cache for bandwidth conservation
resume_support: true # Resumable transfers
rate_limit: 10485760 # 10MB/s rate limit
retry_attempts: 3 # Retry attempts
retry_delay: 5000 # Initial retry delay
temp_dir: '/.doh/temp/transfers' # Staging directory
progress_interval: 1000 # Progress updates (1s)
cleanup_interval: 3600000 # Cleanup interval (1h)
max_session_age: 86400000 # Max session age (24h)
Cross-links: Cloud Anchoring → MCP Integration; Remote Command System → Smart Sync and Git; Connection Management → Error Handling.
Web UI
http://your-instance/admin/cloud-connectCLI
doh cloud anchor
doh cloud anchor-as user@example.com
Programmatic
const isAnchored = await Doh.CloudAnchoring.isInstanceAnchored();
const res1 = await Doh.CloudAnchoring.performCloudAnchoring(username, password);
const res2 = await Doh.CloudAnchoring.performCloudAnchoringAs(requestingUsername, requestingPassword, targetUserEmail);
await Doh.CloudAnchoring.clearSiteAuthToken();
POST /admin/cloud-connect
{ "username": "user@example.com", "password": "your-password" }
Response
{ "success": true, "message": "Instance successfully anchored to Doh Cloud account", "anchored": true }
POST /admin/cloud-connect/clear
{}
Response
{ "success": true, "message": "Cloud anchoring and persistent fingerprint cleared successfully" }
GET /admin/cloud-connect returns the HTML management UI.
See also: CLI commands and API reference.
MCP adds developer tooling on top of Managed Site. There are two distinct roles:
doh mcp. If this machine is cloud‑anchored, it also connects to the cloud manager and gains access to your controllable remote instances.cloud.mcp_controllable: true. These instances appear in cloud tools and accept MCP commands relayed via the cloud.Key behaviors
mcp_controllable to use cloud tools; it is native to your MCP client.doh mcp on a cloud‑anchored machine will connect that MCP client to the cloud automatically (using the managed_site token), enabling access to your controllable instances.Enable remote control explicitly on a target instance
cloud:
mcp_controllable: true
How it works
doh mcp on your development machine. If anchored, it also connects to the cloud as an MCP client.cloud.mcp_controllable: true to be listed and controllable.mcp_command events over their existing cloud socket; handlers execute and return results.Supported MCP commands
| Command | Description |
|---|---|
mcp:restart_instance |
Restart instance with configurable delay |
mcp:get_logs |
Retrieve recent log lines |
mcp:get_status |
System status (memory, CPU, uptime, disk) |
mcp:sync_files |
Upload/download files |
mcp:run_command |
Execute shell command with timeout |
Registered capabilities
logs - Log streamingrestart - Instance restartfilesync - File synchronizationcommands - Shell command executionstatus - Status informationgit - Git operationsSecurity
mcp_controllable: true required)See: MCP_CLOUD_INTEGRATION.md.
When a managed instance is configured with cloud.reverse_proxy.enabled: true, the cloud manager acts as a reverse proxy, allowing external clients to access the instance without direct network exposure.
The cloud manager creates an HTTPS server with Socket.IO support on the configured external_port for each proxied instance. All HTTP requests and Socket.IO events are forwarded to the managed instance via the existing secure WebSocket connection.
HTTP Request Flow:
https://cloud-manager:external_port/pathproxy_http_request event to managed instance via socketSocket.IO Event Flow:
https://cloud-manager:external_portproxy_socket_connect event to managed instanceproxy_socket_event with accumulated cookiesEnable reverse proxy on the managed instance:
cloud:
reverse_proxy:
enabled: true # Enable reverse proxy mode
external_port: 4443 # Port on cloud manager (must be unique per instance)
Requirements:
express_config.ssl_infoexternal_portWhen reverse proxy is enabled, Doh.hostUrlString() automatically returns the cloud manager URL:
// Without reverse proxy
Doh.hostUrlString() // → 'https://localhost:3000'
// With reverse proxy enabled
Doh.hostUrlString() // → 'https://cloud-manager.com:4443'
This ensures Socket.IO clients and API calls automatically connect through the proxy.
For HTTP requests, the managed instance creates full Express-compatible mock objects:
Mock Request Features:
Mock Response Features:
status(), setHeader(), send(), json()Socket.IO events proxied through the cloud manager include cookie forwarding for authentication:
proxy_socket_eventaccessTokenreq.user is attached for authorization checkssetCookies to update client cookiesThis ensures authentication state is preserved across proxied Socket.IO events, enabling secure, stateful connections through the proxy layer.
The cloud manager can pause and resume reverse proxies without disconnecting the instance. This allows temporary deactivation of external access while maintaining the instance connection.
From the Cloud Manager Dashboard:
How It Works:
/.doh/static/cloud_proxies.yaml on the cloud manager (git-ignored)Benefits:
Core commands
| Command | Description | Security |
|---|---|---|
ping |
Connectivity test | Low |
get_status |
System status | Low |
update_file |
File modification | High |
sync_files |
Bulk updates | High |
sync_folder |
Folder sync | High |
enhanced_sync_files |
Sync + dohball processing | High |
restart_service |
Service restart | High |
get_logs |
Log retrieval | Medium |
discover_git_repos |
Find Git repos | Low |
get_git_status |
Git status | Low |
git_stage_files, git_commit, git_pull, git_push |
Git ops | High |
Filesystem and info
| Command | Description |
|---|---|
get_browse |
Browse folders/files with metadata and safe path validation |
read_file_content |
Read a file (auto text/binary), with checksum and metadata |
get_recursive_files |
Efficient recursive file list with exclude patterns |
get_instance_info |
Detailed instance/process/runtime information |
get_available_folders |
Discover available folders from a base path |
get_instance_id |
Fetch the instance fingerprint/id |
delete_path |
Delete file or folder (with safety checks) |
cleanup |
Clean up temp files and stale sessions |
SSH commands
| Command | Description | Security |
|---|---|---|
tty_create |
Create terminal session | High |
tty_data |
Send/receive data | High |
tty_resize |
Resize terminal | Low |
tty_close |
Close terminal | Low |
Transfer commands
| Command | Description | Features |
|---|---|---|
start_chunked_transfer |
Init transfer | Progress, checksums |
upload_chunk |
Upload chunk | Resume, integrity |
download_chunk |
Download chunk | Rate limit, concurrency |
finalize_transfer |
Complete transfer | Verification, dohball cleanup |
get_transfer_status |
Progress | Real-time updates |
cancel_transfer |
Cancel | Safe cleanup |
resume_transfer |
Resume | From checkpoint |
check_file_sync_status |
Sync status | Always-fresh MD5 |
calculate_file_md5 |
MD5 | Single/batch |
Comparison and cross‑instance sync
| Command | Description |
|---|---|
get_file_comparison_data |
Single‑call metadata collector (size, mtime, MD5) for directories |
compare_files_cross_instance |
Server‑side compare between instances (file or folder) |
execute_sync_from_comparison |
Execute a sync from a prior comparison result |
sync_file |
Compare+sync a single file between instances |
sync_files |
Compare+sync a list of files between instances |
sync_folder |
Compare+sync entire folder trees (with dohball cleanup on target) |
Database commands
| Command | Description | Security |
|---|---|---|
list_databases |
List all Dataforge databases | Medium |
get_database_tables |
Get tables in a database | Medium |
checkpoint_database |
Create WAL checkpoint | Medium |
dataforge_select |
Query data with SQL | Medium |
dataforge_export_table |
Export table as JSON | Medium |
get_sync_changes |
Get sync tracking changes | Medium |
apply_sync_changes |
Apply remote sync changes | High |
get_table_schema |
Get table schema info | Low |
browse_table |
Browse table data with pagination | Medium |
delete_table_row |
Delete specific row by ID | High |
truncate_table |
Clear all rows from table | High |
Backup commands
| Command | Description | Security |
|---|---|---|
create_backup |
Create tar.gz database backup | High |
list_backups |
List all backups with metadata | Low |
delete_backup |
Delete specific backup | High |
receive_backup |
Receive backup from peer instance | High |
restore_backup |
Restore database from backup | High |
mark_backup_warehoused |
Mark backup for warehouse storage | Medium |
delete_mirrored_backup |
Delete mirrored backup copy | High |
get_backup_config |
Get backup configuration | Low |
save_backup_config |
Update backup settings | Medium |
check_backup_exists |
Verify backup file exists | Low |
SSL commands (conditional: requires greenlockcli)
| Command | Description | Security |
|---|---|---|
get_ssl_status |
Get certificate status and expiry | Low |
ssl_force_renewal |
Force certificate renewal | High |
ssl_toggle_staging |
Toggle staging/production mode | High |
ssl_toggle_auto_https |
Toggle Auto-HTTPS (try_secure) | High |
Redbird commands (conditional: requires redbird)
| Command | Description | Security |
|---|---|---|
get_redbird_status |
Get proxy configuration and site statuses | Low |
redbird_add_site |
Add site to proxy | High |
redbird_update_site |
Update site configuration | High |
redbird_remove_site |
Remove site from proxy | High |
redbird_reload |
Force configuration reload | Medium |
Command execution flow
Deep dives: Smart Sync, Git Commands, SSH Terminal.
How it works
Key features and benefits
Dohball cleanup
dohball.json in target directoriesremovals entries (caret paths resolved via DohPath)Optimized comparison command: get_file_comparison_data
{ "path": "/path/to/dir", "excludePatterns": [".git/", ".doh/", "node_modules/"] }
Smart sync commands
check_file_sync_statuscalculate_file_md5 (single/batch)Security features: path validation, command allowlist, audit logs, repo validation, timeouts, error classification.
Repository discovery and management
discover_git_reposget_git_status (detailed branch/status/remotes/commits)git_status_raw (raw git output)File operations
git_stage_files, git_unstage_files, git_commitRemote operations
git_pull, git_pushBranch and info
git_branch_list, git_diff, git_initUtilities
get_git_audit_log (operation history)clear_git_caches (force refresh)Caching and performance
Remote database management for Dataforge databases via the cloud connection.
// List all databases
{ type: 'list_databases' }
// Response: { databases: ['app', 'users', ...] }
// Get tables in a database
{ type: 'get_database_tables', database: 'app' }
// Response: { tables: ['app.users', 'app.sessions', ...] }
// Browse table with pagination
{ type: 'browse_table', database: 'app', table: 'app.users', page: 1, pageSize: 50 }
// Response: { rows: [...], total: 150, page: 1, pageSize: 50 }
// Query with SQL
{ type: 'dataforge_select', database: 'app', query: 'SELECT * FROM app.users WHERE active = ?', params: [true] }
// Get table schema
{ type: 'get_table_schema', database: 'app', table: 'app.users' }
// Response: { columns: [{ name: 'id', type: 'TEXT', ... }] }
// Export table as JSON
{ type: 'dataforge_export_table', database: 'app', table: 'app.users' }
// Response: { data: [...], rowCount: 150 }
// Get sync changes since timestamp
{ type: 'get_sync_changes', database: 'app', since: '2024-01-01T00:00:00Z' }
// Apply sync changes from another instance
{ type: 'apply_sync_changes', database: 'app', changes: [...] }
// Delete row (requires ID)
{ type: 'delete_table_row', database: 'app', table: 'app.users', id: 'user123' }
// Truncate table (DANGER: removes all rows)
{ type: 'truncate_table', database: 'app', table: 'app.sessions' }
// Create WAL checkpoint
{ type: 'checkpoint_database', database: 'app' }
Automated database backup management with retention policies, cross-instance mirroring, and warehouse storage.
// Create backup (tar.gz of database files)
{ type: 'create_backup', database: 'app', description: 'Pre-deployment backup' }
// Response: { success: true, backup: { id: 'backup_20240115_120000', size: 1234567, ... } }
// List all backups
{ type: 'list_backups' }
// Response: { backups: [{ id, database, size, createdAt, description, warehoused, ... }] }
// Check if specific backup exists
{ type: 'check_backup_exists', backupId: 'backup_20240115_120000' }
// Delete backup
{ type: 'delete_backup', backupId: 'backup_20240115_120000' }
// Restore from backup (DANGER: replaces current database)
{ type: 'restore_backup', backupId: 'backup_20240115_120000' }
// Receive backup from peer instance
{ type: 'receive_backup', backupData: '...base64...', metadata: {...} }
// Mark backup as warehoused (stored externally)
{ type: 'mark_backup_warehoused', backupId: 'backup_20240115_120000' }
// Delete mirrored copy
{ type: 'delete_mirrored_backup', backupId: 'backup_20240115_120000' }
// Get backup configuration
{ type: 'get_backup_config' }
// Response: { retention_days: 30, auto_backup: true, schedule: '0 2 * * *', ... }
// Update backup settings
{ type: 'save_backup_config', config: { retention_days: 60, auto_backup: true } }
Conditional: Only available when greenlockcli module is loaded.
Provides remote SSL certificate management through the cloud connection.
{ type: 'get_ssl_status' }
// Response:
{
greenlock_available: true,
email: 'admin@example.com',
staging: false,
try_secure: true,
domains: ['example.com'],
certificates: [{
domain: 'example.com',
cert_exists: true,
expires_at: '2024-04-15T00:00:00Z',
days_until_expiry: 45,
needs_renewal: false,
fingerprint: 'AA:BB:CC:...'
}]
}
// Force certificate renewal
{ type: 'ssl_force_renewal' }
// Response: { success: true, message: 'Certificate renewal initiated' }
// Toggle staging/production mode (triggers server restart)
{ type: 'ssl_toggle_staging' }
// Response: { success: true, newMode: 'staging', restart_pending: true }
// Toggle Auto-HTTPS (triggers server restart)
{ type: 'ssl_toggle_auto_https' }
// Response: { success: true, newMode: 'enabled', restart_pending: true }
Note: Toggle commands modify pod.yaml and trigger a server restart.
Conditional: Only available when redbird module is loaded.
Provides remote management of Redbird reverse proxy sites through the cloud connection.
{ type: 'get_redbird_status' }
// Response:
{
redbird_available: true,
port: 80,
ssl: { port: 443 },
sites: [{
path: '/path/to/site',
domain: 'site.example.com',
aliases: ['www.site.example.com'],
status: 'configured', // 'configured' | 'pending' | 'missing_pod' | 'error'
http_port: 3000,
ssl_port: 3443,
try_secure: true
}]
}
// Add site to proxy
{ type: 'redbird_add_site', path: '/path/to/site', domain: 'site.example.com', aliases: ['www.site.example.com'] }
// Update site configuration
{ type: 'redbird_update_site', path: '/path/to/site', domain: 'newdomain.example.com' }
// Remove site from proxy
{ type: 'redbird_remove_site', path: '/path/to/site' }
// Force reload configuration
{ type: 'redbird_reload' }
Behavior:
pod.yaml immediatelycompiled.pod.yaml for port and domain infoZero-config, browser-based terminals via the Cloud Dashboard with multi-session, ANSI/UTF-8 support, resizing, copy/paste, scrollback, dark theme. Requires command:cloud_instance.
States: disconnected, connecting, connected, authenticating, authenticated, reconnecting, error.
Automatic reconnection (exponential backoff)
const delay = Math.min(baseInterval * Math.pow(2, attemptNumber), maxDelay);
Health monitoring: heartbeat (1m), health checks (2m), auto-recovery, error tracking.
Classification: NETWORK, AUTH, FILESYSTEM, CORRUPTION, TIMEOUT, PERMISSION, RESOURCE.
Circuit breakers: network (5/1m), file (3/30s), auth (3/2m).
Friendly messages (examples): ENOTFOUND, Authentication failed, Token expired.
doh cloud endpoint <url>
doh cloud anchor
doh cloud anchor-as <user@example.com>
doh cloud
Features: interactive prompts, spinners, re-anchoring, anchor-as validation, robust errors, token validation.
Status endpoint
GET /api/managed_site/status
{ "success": true, "status": "online", "fingerprint": "abc123", "cloud": { "connected": true } }
Cloud anchoring endpoints: see Cloud Anchoring section.
Doh.CloudAnchoring)isInstanceAnchored() - Check if instance is anchoredperformCloudAnchoring(username, password) - Anchor to own accountperformCloudAnchoringAs(requesterUsername, requesterPassword, targetEmail) - Anchor on behalf of userclearSiteAuthToken() - Remove anchoringvalidateAndRefreshToken() - Validate and refresh JWTglobal.managedSite)establishCloudConnection() - Connect to cloud managerdisconnectFromCloud() - Disconnect from cloudgetCloudConnectionStatus() - Get connection statestoreSiteAuthToken(token) - Store authentication tokenloadSiteAuthToken() - Load stored tokenclearSiteAuthToken() - Clear stored tokengetTransferSessions() - List active transfer sessionscreateTransferSession() - Create new transfer sessioncleanupTransferSession(sessionId) - Clean up sessionperformAdvancedHealthCheck() - Comprehensive health checkcheckCloudConnectionHealth() - Cloud connection healthcheckFileSystemHealth() - File system healthexecuteWithRecovery(fn) - Execute with auto-recoveryattemptAutoRecovery() - Trigger recoverycheckFileSyncStatus(filePath, targetMd5) - Check if file needs synccalculateMd5Checksum(filePath) - Calculate MD5calculateBatchMd5Checksums(filePaths) - Batch MD5 calculationperformSmartSync(sourceDir, targetDir) - Full smart syncmcp_connection)initializeMcpConnection() - Initialize MCP connectiondisconnectFromMcp() - Disconnect MCPgetMcpConnectionStatus() - Get MCP statusisMcpConnected() - Check if MCP connectedThe main module loads core and conditional handlers:
Doh.Module('managed_site', [
'express_router_as_library',
'express_server_as_library',
'nodejs?? import fsE from "fs-extra"',
'nodejs?? CryptoAPI',
'nodejs?? os',
'nodejs?? import http from "http"',
'nodejs?? import { EventEmitter } from "events"',
'nodejs?? import cookie from "cookie"',
'socketio_client',
'managed_site_command_handlers',
'managed_site_command_handlers_git',
'mcp_connection',
'mcp_command_handlers',
'user_host?? user_host'
], function (DohPath, Router, app, fsE, CryptoAPI, os, http, EventEmitter, cookie, ioClient, ...) {
// Conditional handlers loaded dynamically
if (Doh.Packages['greenlockcli']) {
const { sslHandlers } = await Doh.load('managed_site_command_handlers_ssl');
}
if (Doh.Packages['redbird']) {
const { redbirdHandlers } = await Doh.load('managed_site_command_handlers_redbird');
}
});
SSL and Redbird handlers load only when their parent modules are present:
// command_handlers_ssl.js - loads only when greenlockcli is available
Doh.Module('managed_site_command_handlers_ssl', [
'nodejs?? import fsE from "fs-extra"',
'nodejs?? import { execSync } from "child_process"',
'nodejs?? import acme from "acme-client"',
'nodejs?? import crypto from "crypto"',
], async function(DohPath, fsE, execSync, acme, crypto) {
const { greenlockcli } = await Doh.load('greenlockcli');
// Handler implementations...
return { sslHandlers: { ... } };
});
All handler modules export via return value and optionally via Doh.Globals:
Doh.Module('managed_site_command_handlers_example', [...], function(...) {
async function handleMyCommand(payload) {
// Validate payload
if (!payload.requiredField) {
throw new Error('requiredField is required');
}
// Execute operation
const result = await doSomething(payload);
// Return standardized response
return { success: true, data: result };
}
// Export via Doh.Globals for access from main module
Doh.Globals.my_handlers = { handleMyCommand };
// Also return for module consumers
return { myHandlers: { handleMyCommand } };
});
Add new commands in command_handlers.js:
// Add case in the command switch
case 'my_custom_command':
result = await handleMyCustomCommand(payload);
break;
// Implement handler function
async function handleMyCustomCommand(payload) {
const { param1, param2 } = payload;
// Validate inputs
if (!param1) {
return { success: false, error: 'param1 is required' };
}
// Execute with error handling
try {
const result = await myOperation(param1, param2);
return { success: true, data: result };
} catch (error) {
console.error('[CommandHandlers] my_custom_command error:', error);
return { success: false, error: error.message };
}
}
const health = await global.managedSite.performAdvancedHealthCheck();
const cloudHealth = await global.managedSite.checkCloudConnectionHealth();
const fsHealth = await global.managedSite.checkFileSystemHealth();
// MCP connection status
const mcpStatus = mcp_connection.getMcpConnectionStatus();
const mcpConnected = mcp_connection.isMcpConnected();
doh_modules/managed_site/
├── managed_site.js # Main module: connection, auth, proxy handling
├── command_handlers.js # Core + database + backup commands (~6700 lines)
├── command_handlers_git.js # Git operations (~870 lines)
├── command_handlers_ssl.js # SSL/Let's Encrypt (conditional: greenlockcli)
├── command_handlers_redbird.js # Redbird proxy management (conditional: redbird)
├── mcp_connection.js # MCP cloud connection management
├── mcp_command_handlers.js # MCP command execution
├── cloud_anchor_cli.js # CLI implementation
├── cloud_connect_styles.css # Admin UI styling
├── dohball.json # Module cleanup configuration
└── README.md
Credentials: cleared from memory; tokens stored 0600.
Network: HTTPS/WSS; token auth; no sensitive error leakage.
Files: path validation; backups; audit logs.
Errors: no internals; circuit breakers; rate limiting.
Instance not connecting
/admin/cloud-connect 2) Verify endpoint 3) Network reachability 4) Token validityAuth failures
Commands not executing
Debug commands
curl http://localhost:3000/api/managed_site/status
tail -f .doh/logs/doh.log | grep ManagedSite
../cloud_manager/README.md)../mcp/MCP_CLOUD_INTEGRATION.md)../greenlockcli/README.md) - SSL certificate management../redbird/README.md) - Reverse proxy configurationStatus: Production Ready | Commands: 70+ | Conditional modules: SSL (greenlockcli), Redbird (redbird)