planet/docs/technical/en/ops-planet-sh-startup.md

planet.sh Startup Performance Optimization

Background

planet.sh manages start, stop, restart, health checks, and logs for all local services. The previous implementation had several startup issues:

1. AI Provider rebuilt every time, even when code had not changed.
2. Port cleanup could wait up to 45 seconds.
3. Port bind detection used a Python subprocess, adding about 300 ms per call.
4. Plain restart and restart -b behaved differently.

Issue 1: AI Provider Rebuilt Every Time

Root Cause

The build stamp file lived under /tmp/. After WSL or Linux restart, /tmp is cleared, so the stamp_non_empty condition failed and the script decided to rebuild:

# All three conditions had to be true to skip rebuild
image_exists AND stamp_non_empty AND fingerprint_match

Fix

The stamp file moved from a temporary location to a persistent cache path:

AI_PROVIDER_BUILD_STAMP_FILE="${XDG_CACHE_HOME:-$HOME/.cache}/planet/aiprovider_build.sha256"

Writing the stamp creates the directory first:

write_ai_provider_build_stamp() {
    mkdir -p "$(dirname "$AI_PROVIDER_BUILD_STAMP_FILE")"
    compute_ai_provider_build_fingerprint > "$AI_PROVIDER_BUILD_STAMP_FILE"
}

Faster Fingerprint

The previous implementation tarred the whole aiprovider/ directory before hashing, which could take seconds in large trees. The new version uses find + stat and reads only file metadata:

compute_ai_provider_build_fingerprint() {
    find aiprovider \
        -type f \
        ! -path '*/__pycache__/*' \
        ! -name '.env' \
        ! -name '.env.*' \
        ! -name '*.pyc' \
        ! -name '*.pyo' \
        | LC_ALL=C sort \
        | xargs -r stat --format="%Y %s %n" 2>/dev/null
    sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
    python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
}

This is roughly 10 times faster for many-small-file workloads while preserving the same practical rebuild signal. .env and .env.* are excluded because runtime model, key, and Base URL changes should not force an image rebuild.

Docker Build Context

AI Provider only needs root pyproject.toml, uv.lock, and aiprovider/ source code. Sending the entire repository as Docker build context wastes time on frontend assets, PDFs, historical data, and Unreal files.

The root .dockerignore now narrows the context:

**

!pyproject.toml
!uv.lock
!aiprovider/
!aiprovider/**

aiprovider/.env
aiprovider/.env.*
!aiprovider/.env.example

The Dockerfile copies only AI Provider inputs:

COPY pyproject.toml uv.lock /app/
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --frozen --no-dev

COPY aiprovider /app/aiprovider

uv sync uses a BuildKit cache mount. The first build may still depend on network speed, but later builds reuse /root/.cache/uv.

Runtime Configuration

Before starting AI Provider, planet.sh generates a current-user runtime env-file and passes it to Compose or the manual docker run fallback. The default path is ${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_runtime.env. Configuration priority:

1. aiprovider/.env
2. simple export AI_...=... or AI_...=... lines from ~/.zshrc

The default parser is static and only covers AI Provider, image, and proxy variables. It avoids executing interactive shell initialization. Complex shell expansion can be enabled explicitly:

PLANET_LOAD_ZSHRC_ENV=source ./planet.sh start -a

To ignore personal shell config during debugging:

PLANET_LOAD_ZSHRC_ENV=0 ./planet.sh start -a

Skip-Rebuild Behavior

When the fingerprint matches, the script skips docker compose build and starts the existing container:

docker start planet_aiprovider

docker stop stops the container without deleting the image. cleanup_exit_containers removes exited containers but not images, so the next docker start can reuse the existing image.

Issue 2: Slow Port Cleanup

Cause

wait_for_port_release could wait up to 45 seconds by default: 15 attempts times 3 seconds.

Fix

Background process cleanup now uses a 3-second timeout: TERM, 1.5 seconds, KILL, 1.5 seconds.

PORT_RELEASE_ATTEMPTS=15
PORT_RELEASE_INTERVAL=0.2

wait_for_port_release "$port" 15 0.2

wait_for_port_release accepts optional parameters so different situations can choose different timeouts.

Issue 3: Port Detection Used Python

Cause

can_bind_port used python3 -c "import socket..."; each call cost about 300 ms.

Fix

Prefer system tools and keep Python as a fallback:

can_bind_port() {
    local port="$1"
    if command -v ss >/dev/null 2>&1; then
        ! ss -tlnH 2>/dev/null | awk '{print $4}' | grep -qE ":${port}$"
        return
    fi
    if command -v lsof >/dev/null 2>&1; then
        [ -z "$(lsof -tiTCP:"${port}" -sTCP:LISTEN 2>/dev/null)" ]
        return
    fi
    python3 - "$port" <<'PY'
import sys, socket
p = int(sys.argv[1])
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
    s.bind(("", p)); s.close(); sys.exit(0)
except OSError:
    sys.exit(1)
PY
}

Frontend startup now has an additional pre-start cleanup retry layer:

• PORT_PRESTART_RETRIES: defaults to 3 attempts.
• PORT_PRESTART_RETRY_INTERVAL: defaults to 2 seconds.

kill_port_if_requested() first cleans listener PIDs visible in the current environment. It only checks for Windows-side listeners when the script detects WSL, no local listener PID is visible, and the port still cannot bind. In that WSL-only path it requests Administrator PowerShell to delete stale portproxy rules, stop services that own the port, or force-stop the owning process. If the administrator request is canceled, or a system service such as iphlpsvc refuses to stop, the script prints the Windows listener details and Administrator PowerShell recovery commands, then stops startup immediately instead of launching the service into the same port error. If the frontend Vite process only discovers Port 3000 is already in use after launch, the script prints the same Windows listener recovery commands. Non-WSL environments do not run the Windows cleanup path. --allow-lan now exposes 3000 / 8000 / 8010 directly and no longer starts an extra Windows forwarding process; old persistent portproxy rules should be removed.

Issue 4: restart Behavior

Before the stamp path fix:

• restart -b: stop all services, check fingerprint, rebuild only when needed, then start.
• plain restart: stop all services, then often rebuild AI Provider because /tmp lost the stamp.

After moving the stamp file, plain restart uses the same stop + start behavior and the same fingerprint check as restart -b.

State Files, Logs, and Failed-Start Cleanup

planet.sh no longer writes PID files, logs, or runtime env-files to fixed /tmp/planet_* paths. The default state directory is:

${XDG_STATE_HOME:-$HOME/.local/state}/planet

At startup the script creates this directory and tries to set it to 700. The current files include:

• backend.pid / frontend.pid / motion_agent.pid
• backend.log / frontend.log / motion_agent.log
• aiprovider_build.log
• aiprovider_runtime.env
• ports.env

PID writes validate that the PID is a positive integer, include a trailing newline, and try to set file mode 600. PID reads ignore invalid content instead of passing it to kill.

After a successful start, the script records the ports in ports.env. Later ./planet.sh health calls prefer the last started ports; if the state file is missing, health checks fall back to the defaults 8000, 3000, 8010, and 8765. This avoids checking default ports after starting with custom ports.

Startup now has light failed-start cleanup. If start exits before completing, the script only cleans local processes that this run already started: backend, frontend, and Motion Agent. It does not stop services after a successful start. AI Provider, PostgreSQL, and Redis keep their existing container lifecycle behavior.

Health Checks and Hardening

HTTP readiness checks now use curl -fsS --max-time, so 4xx and 5xx responses are no longer treated as healthy.

Process termination now validates:

• signal names are limited to TERM, KILL, INT, and HUP;
• PIDs must be positive integers;
• process group IDs must be positive integers.

This prevents bad PID files or invalid signals from reaching kill.

Frontend and Motion Agent startup failures now call print_port_listener_details(), matching backend port diagnostics. The Windows-side listener and cleanup path still only runs when WSL is detected.

Cross-Platform Notes

The script is currently Linux-first with WSL enhancements. Normal Linux runs do not execute the PowerShell path; WSL gets extra Windows listener, portproxy, and camera guidance.

To make this single script fully portable across Linux, macOS, and WSL, the remaining platform differences should be wrapped behind compatibility helpers:

• stat --format, sort -V, and xargs -r are GNU-style and are not fully compatible with default macOS BSD tools.
• hostname -I, ss, fuser, and systemctl are usually unavailable on macOS.
• tac may be missing on macOS; use awk or Python as a fallback.
• Docker Desktop on macOS does not use systemctl daemon diagnostics.
• Camera auto-detection relies on /dev/video* / v4l2-ctl, which is Linux-specific; macOS should use explicit camera URLs or a separate AVFoundation detector.

The recommended direction is a small platform compatibility layer for port listener detection, version comparison, file metadata, reverse tail, LAN IP discovery, and Docker daemon diagnostics, instead of scattering more platform branches throughout service startup logic.

Production Delivery Boundary

planet.sh is a local development convenience script, not the production startup entrypoint. Production delivery should use Kubernetes Deployment, Service, Ingress, and readiness/liveness probes for ports, health checks, restarts, and rolling upgrades. This removes the need for a host script to reclaim local ports and avoids running the Vite dev server in production.

The production frontend shape is vite build static output served by nginx/Caddy or an equivalent HTTP server. Do not use bun run dev or vite preview in production. The project does not maintain a parallel Webpack build chain; if a future enterprise requirement needs closer Webpack-ecosystem compatibility, run an Rsbuild/Rspack spike first. Electron should only be evaluated when the official target becomes an offline desktop application.

Default Motion Agent Startup

planet.sh now starts the local Motion Agent by default during start and full restart. This makes the Earth page, UE clients, and debug clients able to connect to ws://127.0.0.1:8765/ws/gestures immediately. If the machine has no usable camera, implicit default startup falls back to dry-run protocol mode and does not block backend/frontend startup. Explicit Motion Agent startup through --motion-agent, camera indexes, camera URLs, or WSL USB options still treats live camera failures as real errors.

To skip Motion Agent for this run:

./planet.sh start --non-motion-agent
./planet.sh restart --non-motion-agent

Common options:

• --non-motion-agent: do not start Motion Agent for this start or full restart.
• --motion-agent / -m: explicitly start or restart the Motion Agent for this command; live camera failures are reported as failures.
• --motion-agent-port <port>: override the default WebSocket port 8765.
• --motion-agent-mode <mode>: choose auto, single, dual_redundant, single_fallback, or calibrated_3d; dual is kept as a compatibility alias for redundant dual-camera mode.
• --motion-agent-camera-indexes <indexes>: override auto-detected camera indexes, for example 0 or 0,1. The same can be provided through MOTION_AGENT_CAMERA_INDEXES=0,1.
• --motion-agent-camera-urls <urls>: use RTSP/HTTP camera streams, useful for WSL, phone cameras, or network cameras. The same can be provided through MOTION_AGENT_CAMERA_URLS=....
• --motion-agent-wsl-usbipd: in WSL, try to attach the single detected Windows USB camera through usbipd-win.
• --motion-agent-wsl-usbipd-busid <BUSID>: in WSL, attach the camera matching a usbipd list BUSID; use this when multiple cameras are present.
• --motion-agent-dry-run: start only the protocol service without opening cameras or loading CV dependencies; useful for Web client debugging.

Non-dry-run live mode checks mediapipe and opencv-python before startup. If the current .venv is missing them, the script automatically runs:

uv add mediapipe opencv-python

To disable startup-time Python CV dependency auto-install:

PLANET_MOTION_AGENT_AUTO_INSTALL=0 ./planet.sh start

./planet.sh init also performs a WSL host dependency preflight for usbipd-win. If usbipd.exe is missing, the script first tries winget install -e --id dorssel.usbipd-win, then reuses the repository-bundled dorssel.usbipd-win MSI fallback. If that cached MSI is missing or an architecture-specific MSI is needed, it downloads one and requests an Administrator PowerShell installation. This is best-effort: failure prints next steps but does not block normal initialization. Use ./planet.sh init --non-motion-agent to skip this preflight.

Live mode auto-detects /dev/video*, then prefers an OpenCV probe to keep only indexes that can open and return frames before passing them to the Motion Agent. In WSL/USB camera setups, one camera can expose multiple /dev/video* nodes, and some of them are metadata or non-capture nodes; the script skips those unreadable indexes. In WSL, Windows cameras usually do not appear as /dev/video* automatically. Check available devices first:

Live capture defaults to low-latency settings: 640x360 input and roughly 15Hz recognition events. The worker uses latest-frame reader threads and keeps only the newest frame from each camera, so a slow MediaPipe frame does not make the recognizer drain stale camera backlog. The skeleton debug stream is disabled by default and is only sent at roughly 8Hz while the Earth motion debug panel is open, so normal gesture control is not slowed down by debug data. Status events report both capture FPS and recognition FPS to separate camera throughput issues from recognition cost.

ls /dev/video*

To override auto-detection, pass indexes explicitly:

./planet.sh start --motion-agent --motion-agent-camera-indexes 1,2

In WSL, the more general path is to connect a phone or network camera through an RTSP/HTTP stream:

./planet.sh start --motion-agent --motion-agent-camera-urls http://192.168.1.20:8080/video

To use a Windows USB camera directly from WSL, let the script call usbipd-win. This is opt-in because an attached camera is usually temporarily unavailable to Windows apps while WSL owns it.

When there is only one camera:

./planet.sh start --motion-agent --motion-agent-wsl-usbipd

When there are multiple cameras, inspect the BUSID first and pass it explicitly:

usbipd.exe list
./planet.sh start --motion-agent --motion-agent-wsl-usbipd-busid 3-2

If usbipd attach says the device is not shared or bound, the script tries to open an Administrator PowerShell to run usbipd bind, then retries attach. If UAC is canceled or automatic bind fails, run this manually from an Administrator PowerShell:

usbipd bind --busid 3-2
usbipd attach --wsl --busid 3-2

If WSL has no /dev/video* and no --motion-agent-camera-urls is provided, implicit default startup falls back to dry-run. Explicit live startup stops and prints guidance. Choose one of:

./planet.sh start --motion-agent --motion-agent-camera-urls http://<phone-ip>:8080/video
./planet.sh start --motion-agent --motion-agent-wsl-usbipd
./planet.sh start --motion-agent --motion-agent-dry-run

For explicit live startup, automatic dry-run fallback only happens when PLANET_MOTION_AGENT_WSL_ALLOW_DRY_RUN_FALLBACK=1 is set.

--non-motion-agent is the command-level opt-out. Environment variables can still tune how the service starts:

MOTION_AGENT_DRY_RUN=1 ./planet.sh start

Logs:

./planet.sh log -m

To expose it together with the frontend on the LAN:

./planet.sh start --allow-lan --motion-agent

In this mode the Motion Agent binds 0.0.0.0, and startup output prints both the local WebSocket URL and the recommended LAN WebSocket URL. When opening Earth from another LAN browser, point motionAgent at the display machine:

http://<LAN_IP>:3000/earth?motion=1&motionAgent=ws://<LAN_IP>:8765/ws/gestures

./planet.sh stop also stops a script-managed Motion Agent. ./planet.sh health reports its online/offline status. The Earth page still requires ?motion=1 or browser local storage to enable the Web client connection explicitly.

For ordinary web, WSL, or no-install demo scenarios, you can skip Motion Agent entirely: choose the Browser Camera input source in Earth settings and enable Motion Debug Mode. This route uses browser getUserMedia, so the page must run on HTTPS or localhost and the user must grant camera permission.

Other Cleanup

Two redundant sleep 3 waits were removed because health checks already cover the same readiness:

• start_backend_service: post-database-health-check sleep.
• restart_database_service: post-restart sleep.

Related Files

• planet.sh
• .dockerignore
• aiprovider/Dockerfile
• docker-compose.yml
• docker-compose.simple.yml
• compute_aiprovider_dependency_fingerprint.py