release: bump version to 0.49.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
205
docs/technical/en/ops-planet-sh-startup.md
Normal file
205
docs/technical/en/ops-planet-sh-startup.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# `planet.sh` Startup Performance Optimization
|
||||
|
||||
## Background
|
||||
|
||||
`planet.sh` manages start, stop, restart, health checks, and logs for all local services. The previous implementation had several startup issues:
|
||||
|
||||
1. AI Provider rebuilt every time, even when code had not changed.
|
||||
2. Port cleanup could wait up to 45 seconds.
|
||||
3. Port bind detection used a Python subprocess, adding about 300 ms per call.
|
||||
4. Plain `restart` and `restart -b` behaved differently.
|
||||
|
||||
## Issue 1: AI Provider Rebuilt Every Time
|
||||
|
||||
### Root Cause
|
||||
|
||||
The build stamp file lived under `/tmp/`. After WSL or Linux restart, `/tmp` is cleared, so the `stamp_non_empty` condition failed and the script decided to rebuild:
|
||||
|
||||
```bash
|
||||
# All three conditions had to be true to skip rebuild
|
||||
image_exists AND stamp_non_empty AND fingerprint_match
|
||||
```
|
||||
|
||||
### Fix
|
||||
|
||||
The stamp file moved to a persistent cache path:
|
||||
|
||||
```bash
|
||||
AI_PROVIDER_BUILD_STAMP_FILE="$HOME/.cache/planet/aiprovider_build.sha256"
|
||||
```
|
||||
|
||||
Writing the stamp creates the directory first:
|
||||
|
||||
```bash
|
||||
write_ai_provider_build_stamp() {
|
||||
mkdir -p "$(dirname "$AI_PROVIDER_BUILD_STAMP_FILE")"
|
||||
compute_ai_provider_build_fingerprint > "$AI_PROVIDER_BUILD_STAMP_FILE"
|
||||
}
|
||||
```
|
||||
|
||||
### Faster Fingerprint
|
||||
|
||||
The previous implementation tarred the whole `aiprovider/` directory before hashing, which could take seconds in large trees. The new version uses `find + stat` and reads only file metadata:
|
||||
|
||||
```bash
|
||||
compute_ai_provider_build_fingerprint() {
|
||||
find aiprovider \
|
||||
-type f \
|
||||
! -path '*/__pycache__/*' \
|
||||
! -name '.env' \
|
||||
! -name '.env.*' \
|
||||
! -name '*.pyc' \
|
||||
! -name '*.pyo' \
|
||||
| LC_ALL=C sort \
|
||||
| xargs -r stat --format="%Y %s %n" 2>/dev/null
|
||||
sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
|
||||
python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
|
||||
}
|
||||
```
|
||||
|
||||
This is roughly 10 times faster for many-small-file workloads while preserving the same practical rebuild signal. `.env` and `.env.*` are excluded because runtime model, key, and Base URL changes should not force an image rebuild.
|
||||
|
||||
### Docker Build Context
|
||||
|
||||
AI Provider only needs root `pyproject.toml`, `uv.lock`, and `aiprovider/` source code. Sending the entire repository as Docker build context wastes time on frontend assets, PDFs, historical data, and Unreal files.
|
||||
|
||||
The root `.dockerignore` now narrows the context:
|
||||
|
||||
```dockerignore
|
||||
**
|
||||
|
||||
!pyproject.toml
|
||||
!uv.lock
|
||||
!aiprovider/
|
||||
!aiprovider/**
|
||||
|
||||
aiprovider/.env
|
||||
aiprovider/.env.*
|
||||
!aiprovider/.env.example
|
||||
```
|
||||
|
||||
The Dockerfile copies only AI Provider inputs:
|
||||
|
||||
```dockerfile
|
||||
COPY pyproject.toml uv.lock /app/
|
||||
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
uv sync --frozen --no-dev
|
||||
|
||||
COPY aiprovider /app/aiprovider
|
||||
```
|
||||
|
||||
`uv sync` uses a BuildKit cache mount. The first build may still depend on network speed, but later builds reuse `/root/.cache/uv`.
|
||||
|
||||
### Runtime Configuration
|
||||
|
||||
Before starting AI Provider, `planet.sh` generates a temporary env-file and passes it to Compose or the manual `docker run` fallback. Configuration priority:
|
||||
|
||||
1. `aiprovider/.env`
|
||||
2. simple `export AI_...=...` or `AI_...=...` lines from `~/.zshrc`
|
||||
|
||||
The default parser is static and only covers AI Provider, image, and proxy variables. It avoids executing interactive shell initialization. Complex shell expansion can be enabled explicitly:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=source ./planet.sh start -a
|
||||
```
|
||||
|
||||
To ignore personal shell config during debugging:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=0 ./planet.sh start -a
|
||||
```
|
||||
|
||||
### Skip-Rebuild Behavior
|
||||
|
||||
When the fingerprint matches, the script skips `docker compose build` and starts the existing container:
|
||||
|
||||
```bash
|
||||
docker start planet_aiprovider
|
||||
```
|
||||
|
||||
`docker stop` stops the container without deleting the image. `cleanup_exit_containers` removes exited containers but not images, so the next `docker start` can reuse the existing image.
|
||||
|
||||
## Issue 2: Slow Port Cleanup
|
||||
|
||||
### Cause
|
||||
|
||||
`wait_for_port_release` could wait up to 45 seconds by default: 15 attempts times 3 seconds.
|
||||
|
||||
### Fix
|
||||
|
||||
Background process cleanup now uses a 3-second timeout: TERM, 1.5 seconds, KILL, 1.5 seconds.
|
||||
|
||||
```bash
|
||||
PORT_RELEASE_ATTEMPTS=15
|
||||
PORT_RELEASE_INTERVAL=0.2
|
||||
|
||||
wait_for_port_release "$port" 15 0.2
|
||||
```
|
||||
|
||||
`wait_for_port_release` accepts optional parameters so different situations can choose different timeouts.
|
||||
|
||||
## Issue 3: Port Detection Used Python
|
||||
|
||||
### Cause
|
||||
|
||||
`can_bind_port` used `python3 -c "import socket..."`; each call cost about 300 ms.
|
||||
|
||||
### Fix
|
||||
|
||||
Prefer system tools and keep Python as a fallback:
|
||||
|
||||
```bash
|
||||
can_bind_port() {
|
||||
local port="$1"
|
||||
if command -v ss >/dev/null 2>&1; then
|
||||
! ss -tlnH 2>/dev/null | awk '{print $4}' | grep -qE ":${port}$"
|
||||
return
|
||||
fi
|
||||
if command -v lsof >/dev/null 2>&1; then
|
||||
[ -z "$(lsof -tiTCP:"${port}" -sTCP:LISTEN 2>/dev/null)" ]
|
||||
return
|
||||
fi
|
||||
python3 - "$port" <<'PY'
|
||||
import sys, socket
|
||||
p = int(sys.argv[1])
|
||||
s = socket.socket()
|
||||
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
try:
|
||||
s.bind(("", p)); s.close(); sys.exit(0)
|
||||
except OSError:
|
||||
sys.exit(1)
|
||||
PY
|
||||
}
|
||||
```
|
||||
|
||||
Frontend startup now has an additional pre-start cleanup retry layer:
|
||||
|
||||
- `PORT_PRESTART_RETRIES`: defaults to 3 attempts.
|
||||
- `PORT_PRESTART_RETRY_INTERVAL`: defaults to 2 seconds.
|
||||
|
||||
`kill_port_if_requested()` only kills processes when the current environment can identify listening PIDs. If no PID is visible but the port still cannot bind, it logs diagnostics and lets the service startup flow make the final decision. `start_frontend_with_retry()` only enters the pre-cleanup retry path when a listener PID is visible, so the script no longer spends its retry budget repeatedly killing nothing while a host-side or external network namespace is still releasing the port. Seeing "no listener found but port still unavailable" on the first restart usually means the external environment is still releasing the port, not that a local process cleanup loop is useful.
|
||||
|
||||
## Issue 4: `restart` Behavior
|
||||
|
||||
Before the stamp path fix:
|
||||
|
||||
- `restart -b`: stop all services, check fingerprint, rebuild only when needed, then start.
|
||||
- plain `restart`: stop all services, then often rebuild AI Provider because `/tmp` lost the stamp.
|
||||
|
||||
After moving the stamp file, plain `restart` uses the same `stop + start` behavior and the same fingerprint check as `restart -b`.
|
||||
|
||||
## Other Cleanup
|
||||
|
||||
Two redundant `sleep 3` waits were removed because health checks already cover the same readiness:
|
||||
|
||||
- `start_backend_service`: post-database-health-check sleep.
|
||||
- `restart_database_service`: post-restart sleep.
|
||||
|
||||
## Related Files
|
||||
|
||||
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
|
||||
- [.dockerignore](/home/ray/dev/linkong/planet/.dockerignore)
|
||||
- [aiprovider/Dockerfile](/home/ray/dev/linkong/planet/aiprovider/Dockerfile)
|
||||
- [docker-compose.yml](/home/ray/dev/linkong/planet/docker-compose.yml)
|
||||
- [docker-compose.simple.yml](/home/ray/dev/linkong/planet/docker-compose.simple.yml)
|
||||
- [compute_aiprovider_dependency_fingerprint.py](/home/ray/dev/linkong/planet/scripts/compute_aiprovider_dependency_fingerprint.py)
|
||||
Reference in New Issue
Block a user