Переглянути джерело

docs: update CLAUDE.md for clarity and accuracy in project description and commands

Dr-Swopt 1 тиждень тому
батько
коміт
1c43e2baaf
1 змінених файлів з 119 додано та 46 видалено
  1. 119 46
      CLAUDE.md

+ 119 - 46
CLAUDE.md

@@ -4,27 +4,24 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## What This Is
 
-NestJS backend for the PalmOilAI system. Provides server-side YOLOv8 ONNX inference, SQLite history persistence (with disk image archiving), WebSocket gateways for real-time vision streaming and n8n chat proxying, and a process surveillance monitor for n8n and Ollama.
+NestJS backend optimized for ARM/Android (Termux) deployment. Provides server-side YOLOv8 ONNX inference via WASM (single-threaded, no native bindings), SQLite history persistence (sql.js + disk image archiving), a unified FIS-protocol WebSocket gateway, n8n chat proxying, and a process surveillance monitor.
 
-Detection classes: `Ripe`, `Unripe`, `Underripe`, `Overripe`, `Abnormal`, `Empty_Bunch` (MPOB standard).
+Detection classes: `Ripe`, `Unripe`, `Underripe`, `Overripe`, `Abnormal`, `Empty_Bunch` (MPOB standard, indices 0–5).
 
 ## Commands
 
 ```bash
 npm install
-npm run start:dev               # Watch mode dev server (port 3000)
+npm run start:dev               # Watch mode dev server (HTTPS, port 3000)
 npm run start                   # Single-run dev server
 npm run start:prod              # Production (from dist/)
 npm run build                   # Compile → dist/
 npm run lint                    # ESLint with auto-fix
 npm run test                    # Jest unit tests
-npm run test:watch              # Jest watch mode
-npm run test:cov                # Coverage report
 npm run test:e2e                # End-to-end tests
-jest --testPathPattern=palm-oil # Run a single test file
 ```
 
-Set `PORT` env var to override port 3000. Set `N8N_WEBHOOK_URL` in `.env` for chat proxy and agent readiness probing.
+Set `PORT` env var to override port 3000. Set `N8N_WEBHOOK_URL` in `.env` for chat proxy and n8n process discovery.
 
 ## Architecture
 
@@ -32,39 +29,59 @@ Set `PORT` env var to override port 3000. Set `N8N_WEBHOOK_URL` in `.env` for ch
 
 **`PalmOilModule`** (`src/palm-oil/`)
 
-**`ScannerProvider`** — ONNX inference engine loaded on `OnModuleInit`:
-- `preprocess(buffer)` — `sharp` resize to 640×640, strip alpha, convert HWC→CHW, normalize to `[0.0, 1.0]`, output `[1, 3, 640, 640]` tensor
-- `inference(tensor)` — runs `onnxruntime-node` session
-- `postprocess(tensor, origW, origH, threshold=0.25)` — output is `[1, N, 6]` (`x1, y1, x2, y2, confidence, class_index`):
-  - Captures first 5 raw rows as `raw_tensor_sample` for technical evidence
-  - Filters by confidence threshold
+**`OnnxWasmProvider`** (`src/palm-oil/providers/onnx-wasm.provider.ts`) — WASM-based inference engine, `IScannerProvider` implementation:
+- Loaded on `OnModuleInit`; `ort.env.wasm.numThreads = 1` for ARM/Termux safety
+- Model loaded from `best.onnx` in the project root
+- Execution provider: `['wasm']`
+- `preprocess(buffer)` — Jimp decode → resize 640×640 → RGBA→CHW strip → normalize `[0.0, 1.0]` → `[1, 3, 640, 640]` Float32Array
+- `inference(tensor)` — runs ONNX session with `{ images: ortTensor }` input, extracts first output key
+- `postprocess(tensor, origW, origH, threshold=0.25)` — output shape `[1, N, 6]` (`x1, y1, x2, y2, confidence, class_index`):
+  - Captures first 5 rows as `raw_tensor_sample`
+  - Filters by confidence threshold (default 0.25)
   - Scales coords to original image dimensions
   - Maps `class_index` → `MPOB_CLASSES`
   - Sets `is_health_alert` for `Abnormal` | `Empty_Bunch`
 
-**`PalmOilService`** — Orchestrates inference + persistence. `analyzeImage(imageBuffer, filename, batchId?)` pipeline:
-1. Preprocess via `ScannerProvider`
-2. Inference with timing (`inference_ms`, `processing_ms`)
-3. Postprocess → detections + `raw_tensor_sample`
-4. Save image buffer to `archive/` directory on disk
-5. Persist to SQLite (`HistoryEntity`) — **fully awaited before returning** (guarantees DB write before socket emit)
-6. Returns `AnalysisResponse` with Base64 image, `archive_id`, full `technical_evidence` block
+**`PalmOilService`** — Orchestrates inference + persistence. `analyzeImage(imageBuffer, filename?, batchId?)` pipeline:
+1. Measure original image dimensions via Jimp
+2. Preprocess → inference → postprocess via `ScannerProvider`
+3. Capture timing (`inference_ms`, `processing_ms`)
+4. Write image buffer to `archive/` directory on disk
+5. Persist to SQLite (`HistoryEntity`) — **fully awaited before returning** (ordering guarantee: DB write completes before socket emit)
+6. Return `AnalysisResponse` with Base64 image, `archive_id`, full `technical_evidence` block
 
-Additional methods: `getHistory()` (last 50 records), `getRecordByArchiveId()`, `deleteRecord()` (removes DB row + disk image), `clearAllHistory()` (removes all records + disk images).
+Additional methods: `getHistory()` (last 50 records), `getRecordByArchiveId()`, `deleteRecord()` (removes DB row + disk image), `clearAllHistory()`.
 
-**`VisionGateway`** — WebSocket gateway on `/vision` namespace:
-- `vision:analyze` handler: receives `VisionStreamPayload` (`frame: string, sourceLabel?, batchId?`), strips data-URI prefix, decodes Base64 → Buffer, calls `PalmOilService.analyzeImage()`, emits `vision:result` or `vision:error`
-- `chat:send` handler: receives `ChatPayload`, POSTs to `N8N_WEBHOOK_URL` server-to-server (bypasses browser CORS), unwraps array response to first element, emits `chat:result` or `chat:error`
-- Hard rule: accepts raw, uncompressed Base64 strings only — no binary frames, no WebRTC
+**`VisionGateway`** (`src/palm-oil/vision.gateway.ts`) — Unified FIS-protocol WebSocket gateway (root namespace, no prefix):
 
-**`HistoryEntity`** — TypeORM entity fields:
+**Inbound**: single `request` event carrying `FisAppMessage` envelope `{ serviceId, operation, requestId, payload }`.  
+**Outbound**: `response` event with `FisAppResponse` packets (two-packet streaming pattern).
+
+Route table (`${serviceId}:${operation}`):
+
+| Route | Payload | Behavior |
+|---|---|---|
+| `PalmVision:analyze` | `{ frame: string, sourceLabel?, batchId? }` | Base64 decode → Buffer → `PalmOilService.analyzeImage()` → emit two-packet response with `technical_evidence` |
+| `History:getAll` | — | Query SQLite (last 50), emit array response |
+| `History:delete` | `{ archiveId: string }` | Delete record + disk image, emit `{ deleted: boolean }` |
+| `History:clearAll` | — | Wipe all records + images, emit `{ deleted: number }` |
+| `Chat:send` | `{ message: string }` | POST to `N8N_WEBHOOK_URL` server-to-server, unwrap array to first element, emit response |
+| `Chat:clear` | — | Generate new session UUID, emit `{ status: 'success' }` |
+| `PalmHistory:GetImage` | `{ archiveId: string }` | Load image from disk, encode to Base64 data URL, emit response |
+
+Client lifecycle:
+- On connect: push latest surveillance snapshot (if available)
+- Store session UUID in `client.data.sessionId` (used for Chat payloads)
+- Every 500ms tick: broadcasts `response` packet with `SystemMetrics` to all connected clients (via `SurveillanceService` callback registered in `onModuleInit`)
+
+**`HistoryEntity`** (`src/palm-oil/entities/history.entity.ts`) — TypeORM entity fields:
 - `id` (PK auto), `archive_id` (unique), `batch_id` (nullable, indexed), `filename`
 - `total_count`, `industrial_summary` (simple-json), `detections` (simple-json array)
 - `inference_ms`, `processing_ms`, `image_path` (disk reference)
 - `created_at` (auto timestamp)
 
 **`mpob-standards.ts`** (`src/palm-oil/constants/`) — Source of truth:
-- `MPOB_CLASSES`: index 0–5 → class name strings
+- `MPOB_CLASSES`: index → class name (`0: Empty_Bunch`, `1: Underripe`, `2: Abnormal`, `3: Ripe`, `4: Unripe`, `5: Overripe`)
 - `GRADE_COLORS`: class name → hex color
 - `HEALTH_ALERT_CLASSES`: `['Abnormal', 'Empty_Bunch']`
 
@@ -72,32 +89,88 @@ Additional methods: `getHistory()` (last 50 records), `getRecordByArchiveId()`,
 
 **`SurveillanceModule`** (`src/surveillance/`)
 
-**`SurveillanceService`** — Boots on `OnModuleInit`, runs two background loops:
-- **500ms poll loop**: discovers PIDs for n8n (Node.js process containing "n8n" in cmd) and Ollama (`ollama_llama_server` or `ollama`), then calls `pidusage` for CPU/memory metrics
-- **Port-level heartbeat**: n8n PID only included if port 5678 actively accepts TCP connections (800ms timeout); evicts PID if port goes silent or process dies
-- Exposes: `getLatestMetrics()`, callback registration for tick events
+**`SurveillanceService`** — 500ms process monitor via `systeminformation`:
+- `OnModuleInit`: starts polling interval; `OnModuleDestroy`: clears it
+- Per tick: queries CPU load, memory, process list via `systeminformation`
+- Discovers processes by identity:
+  - **NestJS**: match by `process.pid`
+  - **n8n**: command substring match (only if `N8N_WEBHOOK_URL` contains `localhost`)
+  - **Ollama**: command substrings `['ollama', 'ollama_llama_server']`
+- Classifies each as `ACTIVE` (CPU > 1.0%), `IDLE` (CPU ≤ 1.0%), or `OFFLINE` (not found)
+- Stores `SystemMetrics` snapshot; triggers registered callback for `VisionGateway` broadcast
+- Public API: `getLatestMetrics()`, `registerMetricsCallback(cb)`
+
+```typescript
+interface ServiceStatus { service: string; pid: number | null; status: 'ACTIVE'|'IDLE'|'OFFLINE'; cpu: number; memory: number; }
+interface SystemMetrics { cpuLoad: number; memUsed: number; memTotal: number; uptime: number; services: ServiceStatus[]; timestamp: Date; }
+```
 
-**`SurveillanceGateway`** — WebSocket gateway on `/monitor` namespace:
-- Wires `SurveillanceService` callbacks: every 500ms tick broadcasts `monitor:data` to all clients
-- On client connect: immediately pushes latest snapshot (no blank load screen)
-- `monitor:subscribe` handler: acknowledges and re-sends current snapshot
+Note: No separate `SurveillanceGateway`. Surveillance broadcasts are handled inside `VisionGateway` via the registered callback.
 
 ### Database
-SQLite file `palm_history.db` in the project root. Managed by TypeORM with `synchronize: true` (auto-creates/migrates tables — dev only). No external DB setup required.
+
+SQLite file `palm_history.db` in project root. Managed by TypeORM with driver `sqljs` (`sql.js`) and `autoSave: true`. No external DB setup required. `synchronize: true` auto-creates/migrates tables on startup (dev only).
+
+### Key Interfaces (`src/palm-oil/interfaces/`)
+
+```typescript
+interface DetectionResult {
+  bunch_id: number;           // 1-based sequential count
+  class: string;              // MPOB class name
+  confidence: number;         // [0, 1]
+  is_health_alert: boolean;   // true if Abnormal | Empty_Bunch
+  box: [x1, y1, x2, y2];     // pixel coords in original image dims
+}
+
+interface AnalysisResponse {
+  detections: DetectionResult[];
+  archive_id: string;
+  image_base64: string;       // Base64-encoded image
+  technical_evidence: {
+    engine: 'NestJS-ONNX';
+    archive_id: string;
+    total_count: number;
+    threshold: number;        // 0.25
+    industrial_summary: Record<string, number>;
+    raw_tensor_sample: number[][];  // first 5 ONNX output rows
+  };
+  inference_ms: number;
+  processing_ms: number;
+}
+```
 
 ### Required Files
-- `best.onnx` — YOLOv8 ONNX model, must be placed in the **project root** (not `src/`). Loaded by `ScannerProvider` at startup.
-- `archive/` directory — created automatically by `PalmOilService` for disk image storage.
+
+- `best.onnx` — YOLOv8 ONNX model in **project root** (not `src/`). Loaded at startup.
+- `cert/127.0.0.1+1-key.pem` and `cert/127.0.0.1+1.pem` — TLS certificates. HTTPS is always on (no HTTP fallback).
+- `archive/` — Auto-created by `PalmOilService` on first write.
+- `tflite/` — TFLite model files (present but not loaded by current server code).
 
 ### Configuration (`.env`)
+
 ```
-N8N_WEBHOOK_URL=<n8n webhook URL>
-PORT=3000   # optional
+N8N_WEBHOOK_URL=<n8n webhook URL>   # Optional; required for Chat:send and n8n process discovery
+PORT=3000                           # Optional
+HOST=0.0.0.0                        # Default bind address
 ```
 
-### WebSocket Event Contracts
-| Namespace | Client → Server | Payload | Server → Client |
-|---|---|---|---|
-| `/vision` | `vision:analyze` | `{ frame: string, sourceLabel?: string, batchId?: string }` | `vision:result`, `vision:error` |
-| `/vision` | `chat:send` | `{ message: string }` | `chat:result`, `chat:error` |
-| `/monitor` | `monitor:subscribe` | — | `monitor:data` (MonitorPayload[]) |
+### CORS Whitelist
+
+Origins hardcoded in `main.ts`:
+- `https://192.168.100.100:4200`
+- `https://192.168.100.79:4200`
+- `https://localhost:4200`
+
+Update `main.ts` to add new origins (e.g., new Android device IPs).
+
+### Key Differences from `nestjs/` (server-desktop)
+
+| Concern | `nestjs/` | `server-android/` |
+|---|---|---|
+| ONNX runtime | `onnxruntime-node` (native) | `onnxruntime-web` (WASM, `numThreads=1`) |
+| Image processing | `sharp` | `Jimp` |
+| SQLite driver | `sqlite3` | `sql.js` (TypeORM `sqljs`) |
+| WebSocket namespaces | `/vision` + `/monitor` (separate) | Root namespace (unified) |
+| WebSocket protocol | Flat events (`vision:analyze`) | FIS envelope (`request`/`response`) |
+| Surveillance | Separate `SurveillanceGateway` on `/monitor` | Inline broadcast in `VisionGateway` |
+| Process metrics | `pidusage` | `systeminformation` |