Skip to content

Highdra CLI Structure and Web UI Integration

This document explains how highdra-cli is organized, how it uses the database, and how it connects to the Flask-based web UI for remote command execution and software checkout workflows.

High-Level Structure

Core package layout in src/highdra/:

highdra/
  __init__.py                  # package version export
  cli/
    __init__.py                # Typer app setup + command lifecycle + DB run logging
    __main__.py                # python -m highdra.cli
    runner.py                  # starts websocket runner service
    software.py                # starts runner and opens software checkout web UI
    arm.py, battery.py, ...    # hardware command groups
  models/
    __init__.py                # SQLAlchemy engine factory + model exports
    base_entity.py             # schema-aware declarative base
    command_run.py             # command run metadata
    command_log_event.py       # per-line/structured logs
    available_runner.py        # runner registration + heartbeat state
    software_manifest*.py      # software checkout metadata
    root_certificate.py        # CA certificate persistence
  websocket_runner/
    __init__.py                # runner API exports
    runner_service.py          # websocket server + auth + heartbeat lifecycle
    command_executor.py        # command router (ping/status/cli/software_checkout_step)
    command_messages.py        # typed websocket payload/event models
    serial_in_out.py           # terminal stream cleanup and input-request framing
    ws_input.py                # websocket-backed interactive input protocol helpers
    certificate/               # runner TLS certificate creation and bootstrap helpers
        cert_gen.py            # generates leaf certificates for runner TLS
        common.py              # shared certificate/key path and utility helpers
        root_fetch.py          # fetches persisted CA material for TLS setup
    command_handler/
      general_cli.py           # allowlisted generic CLI execution
      software_checkout_step.py# step-wise download/automated/manual execution
  steps/
    software_download.py       # fetches scripts/assets
    software_execute.py        # runs downloaded scripts

Module Initialization

highdra uses explicit module init files to define package boundaries and side effects.

  • src/highdra/__init__.py

    • Keeps the top-level package lightweight.
    • Exposes only __version__.

  • src/highdra/cli/__init__.py

    • Creates the Typer app and registers all subcommands (arm, battery, body, esc, images, config, software, runner).
    • Creates a runtime Session object (ctx.obj) used across commands.
    • Initializes command-run DB tracking when --db-logging is enabled.
    • Finalizes command status (success/failure, exit code, timestamps) in _finalize_command_run().

  • src/highdra/websocket_runner/__init__.py

    • Re-exports runner service and websocket message types (RunnerService, RunnerConfig, RunnerCmdResult, etc.) for stable imports.

Database Usage

Database access is shared between CLI runner processes and the web UI.

Connection and Schema

  • src/highdra/models/__init__.py

    • get_engine() reads PG_ALEMBIC_CONNECTION_STRING and requires PG_SCHEMA_NAME.
    • Returns a cached SQLAlchemy engine.

  • src/highdra/models/base_entity.py

    • Declares metadata = MetaData(schema=os.getenv("PG_SCHEMA_NAME")).
    • All mapped tables default to the configured schema (for example v0).

What Is Persisted

  • CommandRun + CommandLogEvent

    • Command lifecycle and output logs.
    • Written by CLI entrypoint (cli/__init__.py) and consumed by web history pages.

  • AvailableRunner

    • Runner endpoint, auth token, online/enabled flags, heartbeat and claim window.
    • Updated continuously by websocket_runner/runner_service.py.

  • SoftwareManifest and step records

    • Used by web routes to build checkout step payloads.

  • RootCertificate

    • Stores generated CA cert/key material for TLS websocket usage.

Connection to Flask Web UI

The web UI (software_management_web) uses the same DB models and opens browser websocket connections directly to runners.

Browser to Handler Flow

sequenceDiagram
  participant B as Browser
  participant RS as RunnerService
  participant CE as CommandExecutor
  participant H as Handler

  B->>RS: WebSocket connect + auth response
  B->>RS: {command: "cli" | "software_checkout_step"}
  RS->>CE: execute(payload)
  CE->>H: dispatch to specific handler
  H-->>CE: output/result events
  CE-->>RS: command_result
  RS-->>B: streamed output + final result

A. Generic Command-Line Execution via Web

  1. Flask renders runner command page:

    • software_management_web/src/software_management_web/controllers/runners/routes.py
    • software_management_web/src/software_management_web/templates/runners/runner_detail.html

  2. Browser JS opens websocket and sends command payload:

    • software_management_web/src/software_management_web/static/js/runners/runner_detail.js
    • Sends payloads like:
      • { "command": "ping" }
      • { "command": "cli", "cli": "highdra esc checkout" }

  3. highdra-cli runner handles it:

    • src/highdra/websocket_runner/runner_service.py authenticates with HMAC challenge-response.
    • src/highdra/websocket_runner/command_executor.py dispatches to CliCommandHandler.
    • src/highdra/websocket_runner/command_handler/general_cli.py validates allowlisted top-level commands, then executes:
      • python -m highdra.cli ...

  4. Output and prompts are streamed back:

    • Command output events (command_output).
    • Interactive prompts (input_request / input_response).
    • Final result (command_result).

B. Software Checkout Steps via Web

  1. Flask prepares manifest and sorted step payload:

    • software_management_web/src/software_management_web/controllers/commands/routes.py

  2. Page + JS orchestrate step execution:

    • Template: software_management_web/src/software_management_web/templates/commands/software_checkout_run.html
    • JS modules: software_checkout_run.js, software_checkout_run_websocket.js, software_checkout_run_ui.js, software_checkout_run_utils.js

  3. Browser sends per-substep websocket command:

    • { "command": "software_checkout_step", ... }
    • Substeps: download, automated, manual

  4. Runner executes substeps:

    • src/highdra/websocket_runner/command_handler/software_checkout_step.py
    • Downloads scripts/assets, executes automated.py, and streams manual markdown + reference files back to UI.

  5. UI confirms each substep interactively:

  6. Confirmation exchanged via input_request and input_response messages.

TLS/WSS Notes

  • Runner TLS cert setup is performed in:
    • src/highdra/cli/runner.py
    • src/highdra/websocket_runner/runner_service.py
  • Web UI can switch websocket preference (ws:// vs wss://) and provides CA certificate download/install guidance in:
    • software_management_web/src/software_management_web/templates/ca_cert.html
  • User-facing software checkout command docs:
    • docs/commands/software/checkout.md
  • History/logging UI based on CommandRun and CommandLogEvent:
    • software_management_web/src/software_management_web/controllers/history/routes.py