UI-Prototype/CLAUDE.md

CLAUDE.md — frontend

Angular 21 SPA. Multi-tenant dashboard with a lazy-loaded PalmVision sub-app for MPOB-standard palm oil fruit bunch (FFB) ripeness detection. Three inference engines: browser ONNX WASM, browser TFLite WASM, and NestJS remote (WebSocket).

Commands

npm start                   # Dev server at 0.0.0.0:4200
npm run build               # Production build → dist/fisapp-ui
npm run build:dev           # Dev build → dist/dev
npm run build:prod          # Prod build → dist/rc
npm run build:leave:prod    # Leave-module variant
npm run build:quot:prod     # Quotation variant
npm run build:maf:quot:prod # MAF quotation variant
npm run test                # Karma + Jasmine
npm run clean               # Clear Angular + npm caches

Architecture

Application Shell (src/app/)

• app.config.ts — ApplicationConfig: bootstraps NGXS root store (withNgxsStoragePlugin({keys:'*'}), withNgxsReduxDevtoolsPlugin), Angular Router (hash location, onSameUrlNavigation:'reload'), HTTP client, service worker (registered immediately).
• app.routes.ts — Root routes: dashboard, auth, leave, tender, src.palm.vision (all lazy-loaded). PalmVision loads from angularlib/palm-vision/palm-vision.module.
• app.component.ts — Root component: 30-minute session timeout, PWA install prompt, theme management (light/dark + blue/pink), multi-language (en_US, ms_MY, zh_Hans), maintenance mode alert.

Connection Config (src/config/config.json)

Backend URL and connection settings live here:

{
  "connection": {
    "uacp": "https://localhost:3000",
    "uacp_ws": "wss://localhost:3000",
    "uacpEmulation": "on"
  }
}

• uacp — REST/HTTP base URL for the NestJS backend
• uacp_ws — WebSocket URL for Socket.io
• uacpEmulation — "on" enables local buffering mode (dev); "off" = direct publish (prod)

Loaded at runtime via HttpClient.get('/config/config.json') by RemoteInferenceService and DpService.

PalmVision — UI Components (src/dependencies/angularlib/palm-vision/)

Lazy-loaded at /src.palm.vision via PalmVisionModule.

Sub-routes:

Path	Component	Purpose
analyzer (default)	AnalyzerComponent	Inference UI — file/camera input, bounding box canvas
vault	HistoryComponent	Batch history vault with canvas thumbnails
chat	ChatbotComponent	Industrial Intelligence Portal (n8n chat proxy)

AnalyzerComponent — Three-mode inference UI:

• Engine selector: local-onnx (ONNX WASM), local-tflite (TFLite WASM), remote (NestJS server)
• Input: drag-and-drop file zone or live webcam (getUserMedia, environment-facing, 640×640)
• Batch carousel: prev/next navigation across batchFrames
• Canvas overlay via renderPredictionsWithBoxes(frame) — uses norm_box normalized coords preferentially, falls back to box pixel coords
• MPOB color palette: Ripe #4caf50, Unripe #ff9800, Underripe #ffeb3b, Overripe #9c27b0, Abnormal #f44336, Empty_Bunch #607d8b

HistoryComponent — Batch vault:

• Groups records by batch_id into BatchGroup[]
• Canvas thumbnails rendered via renderThumbnailWithBoxes() using norm_box coordinates

BatchReportComponent (history/batch-report/) — Batch detail viewer:

• Scalar metrics: frameCount, totalDetections, avgInferenceMs, avgProcessingMs, batchStart, batchEnd
• classTally breakdown with MPOB color-coded percentage bars
• Frame gallery grid — clicking opens FrameInspectorDialogComponent

FrameInspectorDialogComponent (history/batch-report/frame-inspector-dialog.component.ts) — Inline dialog:

• Canvas capped at 720px; bounding boxes drawn using norm_box preferentially, falls back to box

ChatbotComponent (palm-vision/chatbot/chatbot.component.ts) — Industrial Intelligence Portal chat UI, sends via Chat:send/Chat:clear FIS envelope over the socket (see DpService/NgxSocketService in dp-ui/). Not related to the legacy angularlib/chat/chat.component (HTTP-based, used elsewhere in the app).

• Backend (server-desktop) normalizes the n8n agent's reply into { text, has_visuals, visual_data } before emitting it (see server-desktop/CLAUDE.md).
• chatbot/models/intelligence.model.ts — ChatResponse/ChatVisualData mirror that schema; buildIntelligenceMessage() turns a response into a TextMessage or DataMessage; toChartConfiguration() maps visual_data to a Chart.js ChartConfiguration.
• Chart rendering reuses the in-house <chart> component (angularlib/chart/ChartComponent/ChartModule, wraps chart.js/auto) — no separate charting library. visual_data.type is restricted by the n8n system prompt to bar, line, pie, doughnut, radar, polarArea (the types matching the flat labels[]/datasets[{label,data:number[]}] shape supported by this schema); bubble/scatter are not supported since they need {x,y} point objects.

PalmVision — Logic (src/dependencies/fis-vision/)

Self-contained library: services, NGXS state, and Web Worker for inference.

InferenceService — Local WASM inference:

• analyze(file, batchId?, mode) → Observable<InferenceFrame>
• Preprocesses image → posts tensor to Web Worker → emits result
• preprocessImage(dataUrl) — resizes to 640×640, converts RGBA→CHW, normalizes [0.0, 1.0] → Float32Array [1, 3, 640, 640]

RemoteInferenceService — WebSocket bridge to NestJS backend:

• Loads src/config/config.json for uacp / uacp_ws
• analyze(file, sourceLabel?, batchId?) → Observable<InferenceFrame> via PalmVision:analyze
• getHistory(), getBatchDetails(batchId), deleteRecord(id), clearHistory(), getImage(archiveId), saveExternalResult(payload)

VisionState (NGXS) — Key actions:

Action	Effect
SubmitBatchAnalysis	Fan-out per-file inference; local results persisted via saveExternalResult before vault display
LoadHistory	Fetch last 50 records from SQLite
ToggleBatchGroup	Expand/collapse history accordion
LoadGroupImages	Lazy-load archived images for a batch
DeleteHistoryRecord / ClearAllHistory	Delete one or all records

Web Worker (workers/inference.worker.ts):

• ONNX path: Loads /assets/models/onnx/best.onnx; output shape [1, N, 6]; confidence threshold ≥ 0.25
• TFLite path: Loads /assets/models/tflite/best_float16.tflite; output [1, 300, 6]; confidence ≥ 0.20

Key Interfaces

interface DetectionResult {
  bunch_id: number;
  class: string;              // MPOB class name
  confidence: number;         // [0, 1]
  is_health_alert: boolean;   // true if Abnormal | Empty_Bunch
  box: [x1, y1, x2, y2];     // absolute pixel coords
  norm_box?: [nx1, ny1, nx2, ny2]; // normalized [0,1] coords
}

interface InferenceFrame {
  frameId: string;
  batchId?: string;
  imageDataUrl: string;
  detections: DetectionResult[];
  inference_ms: number;
  processing_ms: number;
  total_count: number;
  industrial_summary: Record<string, number>;
  source: 'wasm-local' | 'remote' | 'n8n';
}

In-Source Dependencies (src/dependencies/)

Folder	Purpose
angularlib/	Shared UI components (forms, dialogs, chat, palm-vision UI)
dp-ui/	DpService, NgxSocketService, DPState, FIS message models
fis/	Domain modules (leave, tender, approval), MetadataState
fis-vision/	PalmVision inference services, NGXS state, Web Worker

fis-commons is CDN-hosted — no local checkout needed.

Models & WASM Assets

• src/assets/models/onnx/best.onnx — YOLOv8 ONNX model
• src/assets/models/tflite/best_float16.tflite — TFLite model
• src/assets/wasm/ — ONNX Runtime Web JS + WASM bundles
• src/assets/tflite-wasm/ — TensorFlow Lite WASM runtime

Multi-Build Strategy

angular.json defines four production configurations that swap menu and assets via fileReplacements:

Config	Output	Swaps
production	dist/rc	—
leave-prod	dist/leave	menu.ts → menu.leave.ts, leave assets
quotation-prod	dist/quotation	menu.ts → menu.quotation.ts, quotation assets
maf-quot-prod	dist/maf/quotation	MAF quotation assets

PWA

Service worker registered immediately (registerImmediately) via ngsw-config.json. PWA install prompt handled in AppComponent (mobile only). Three language packs: en_US, ms_MY, zh_Hans.

CORS / Connection

Frontend connects to NestJS backend at https://localhost:3000 (change in src/config/config.json). Backend CORS whitelist is hardcoded in server-desktop/src/main.ts — add new device IPs there.