Prechádzať zdrojové kódy

fixed studio page. but historical is still faulty

Dr-Swopt 6 dní pred
rodič
commit
8a5f0799e3

+ 43 - 25
CLAUDE.md

@@ -1,6 +1,6 @@
 # CLAUDE.md — frontend
 
-Angular 20 SPA. Multi-tenant dashboard with a lazy-loaded `src.palm.vision` sub-app for MPOB-standard palm oil fruit bunch (FFB) ripeness detection. Three inference engines: browser ONNX WASM, NestJS remote (WebSocket), and n8n edge passthrough.
+Angular 21 SPA. Multi-tenant dashboard with a lazy-loaded `src.palm.vision` sub-app for MPOB-standard palm oil fruit bunch (FFB) ripeness detection. Three inference engines: browser ONNX WASM, browser TFLite WASM, and NestJS remote (WebSocket).
 
 ## Commands
 
@@ -20,9 +20,9 @@ npm run clean               # Clear Angular + npm caches
 
 ### Application Shell (`src/app/`)
 
-- **`app.config.ts`** — `ApplicationConfig`: bootstraps NGXS root store (`withNgxsStoragePlugin`, `withNgxsReduxDevtoolsPlugin`), Angular Router (hash location), HTTP client, service worker.
+- **`app.config.ts`** — `ApplicationConfig`: bootstraps NGXS root store (`withNgxsStoragePlugin({keys:'*'})`, `withNgxsReduxDevtoolsPlugin`), Angular Router (hash location, `onSameUrlNavigation:'reload'`), HTTP client, service worker (registered immediately).
 - **`app.routes.ts`** — Root routes: `dashboard`, `auth`, `leave`, `tender`, `src.palm.vision` (all lazy-loaded except dashboard).
-- **`app.component.ts`** — Root component: session timeout, PWA install prompt, theme management.
+- **`app.component.ts`** — Root component: 30-minute session timeout, PWA install prompt, theme management (light/dark + blue/pink), multi-language (en_US, ms_MY, zh_Hans), maintenance mode alert.
 
 ### PalmVision Sub-App (`src/src.palm.vision/`)
 
@@ -39,10 +39,11 @@ Lazy-loaded at `/src.palm.vision`. Provides its own NGXS `VisionState` via `prov
 #### Services (`src/src.palm.vision/services/`)
 
 **`InferenceService`** — Local WASM inference:
-- `analyze(file, batchId?)` → `Observable<InferenceFrame>` — preprocesses image → posts tensor to Web Worker → emits result
-- `preprocessImage(dataUrl)` — resizes to 640×640, converts RGBA→CHW, normalizes `[0.0, 1.0]`
-- `results$: Subject<InferenceFrame>` — all completed frames
-- `queueDepth$: BehaviorSubject<number>` — pending worker frames
+- `analyze(file, batchId?, mode)` → `Observable<InferenceFrame>` — preprocesses image → posts tensor to Web Worker → emits result
+- `preprocessImage(dataUrl)` — resizes to 640×640 on canvas, converts RGBA→CHW, normalizes `[0.0, 1.0]` → `Float32Array [1, 3, 640, 640]`
+- `results$: Subject<InferenceFrame>` — hot stream of all completed frames
+- `queueDepth$: BehaviorSubject<number>` — pending worker frame count
+- `pendingMap: Map<string, fn>` — in-memory frameId → observer resolver
 
 **`RemoteInferenceService`** — WebSocket bridge to NestJS backend:
 - `analyze(file, sourceLabel?, batchId?)` → `Observable<InferenceFrame>` — sends `PalmVision:analyze` via FIS envelope
@@ -52,6 +53,8 @@ Lazy-loaded at `/src.palm.vision`. Provides its own NGXS `VisionState` via `prov
 - `getImage(archiveId)` → `Observable<{ archiveId, image_data }>` — `PalmHistory:GetImage`
 - `saveExternalResult(payload)` → `Observable<any>` — `PalmHistory:SaveExternalResult`
 
+All calls are wrapped in the FIS envelope and routed via `DpService.stream()`.
+
 Connection config loaded from `src/src.palm.vision/config/config.json`:
 ```json
 { "connection": { "uacp": "http://localhost:3000", "uacp_ws": "ws://localhost:3000/socket.io", "uacpEmulation": "on" } }
@@ -61,7 +64,14 @@ Connection config loaded from `src/src.palm.vision/config/config.json`:
 
 **`VisionStateModel`:**
 ```typescript
-{ items: any[]; loading: boolean; expandedBatchIds: string[]; currentInference: InferenceFrame | null }
+{
+  items: any[];                        // History records from server
+  loading: boolean;
+  expandedBatchIds: string[];          // Expanded accordion groups
+  currentInference: InferenceFrame | null;
+  batchFrames: InferenceFrame[];       // Current batch results
+  selectedFrameIndex: number | null;
+}
 ```
 
 **Selectors:** `VisionState.items`, `VisionState.loading`, `VisionState.expandedBatchIds`, `VisionState.currentInference`
@@ -70,32 +80,40 @@ Connection config loaded from `src/src.palm.vision/config/config.json`:
 
 | Action | Payload | Effect |
 |---|---|---|
-| `SubmitBatchAnalysis` | `{ files: File[]; mode: 'local'\|'remote'\|'n8n' }` | Run batch inference |
+| `SubmitBatchAnalysis` | `{ files: File[]; mode: 'local-onnx' \| 'local-tflite' \| 'remote' }` | Run batch inference; local results persisted via `saveExternalResult` before vault display |
 | `ToggleBatchGroup` | `{ batchId: string }` | Expand/collapse history accordion |
 | `LoadGroupImages` | `{ batchId: string }` | Lazy-load archived images for a batch |
 | `LoadHistory` | — | Fetch last 50 records from SQLite |
 | `DeleteHistoryRecord` | `{ id: string }` | Delete record + disk image |
 | `ClearAllHistory` | — | Wipe all records |
 
+**Key `submitBatchAnalysis` behaviour:** generates a shared `batchId = UUID`, fans out per-file streams via `merge()`, collects via `toArray()`, then dispatches `LoadHistory`. In `local-onnx`/`local-tflite` mode each frame is first persisted to the backend via `saveExternalResult()` (with `timeout(5000)` + `catchError` fallback) before being committed to the batch.
+
 #### Components
 
 **`AnalyzerComponent`** — Three-mode inference UI:
-- Engine selector: `local` (WASM), `remote` (NestJS), `n8n` (edge)
-- Input: drag-and-drop file zone or live webcam (`getUserMedia`, environment-facing)
-- Canvas overlay via `renderPredictionsWithBoxes(frame)` — draws bounding boxes + confidence labels
+- Engine selector: `local-onnx` (ONNX WASM), `local-tflite` (TFLite WASM), `remote` (NestJS server)
+- Input: drag-and-drop file zone or live webcam (`getUserMedia`, environment-facing, 640×640)
+- Batch carousel: prev/next navigation across `batchFrames`
+- Canvas overlay via `renderPredictionsWithBoxes(frame)` — uses `norm_box` normalized coords preferentially, falls back to `box` pixel coords
 - MPOB color palette: `Ripe #4caf50`, `Unripe #ff9800`, `Underripe #ffeb3b`, `Overripe #9c27b0`, `Abnormal #f44336`, `Empty_Bunch #607d8b`
 
 **`HistoryComponent`** — Batch vault:
 - Groups `items` by `batch_id` into `BatchGroup[]` via `combineLatest([items$, expandedBatchIds$])`
 - `@ViewChildren('thumbCanvas')` — canvas thumbnails rendered via `renderThumbnailWithBoxes()` using `norm_box` coordinates
 - `paintAllVisibleCanvases()` with `setTimeout(0)` deferral; `data-archive-id` attribute maps canvas elements to data items
-- Mode badge variants: `local` (WASM), `remote` (Server), `n8n`
+- Mode badge variants: `ONNX`, `TFLite`, `Server`
 
 **`ChatbotComponent`** — Thin wrapper around `angularlib/chat/chat.component` with title "Industrial Intelligence Portal".
 
 #### Web Worker (`src/src.palm.vision/workers/inference.worker.ts`)
 
-Receives `{ frameId, batchId, imageDataUrl, tensor, processingStart }`, runs ONNX inference, returns `InferenceFrame`. Keeps inference off the main thread for WASM local mode.
+Receives `{ frameId, batchId, imageDataUrl, tensor: ArrayBuffer (transferred), processingStart, mode }`.
+
+- **ONNX path (`local-onnx`):** Loads ONNX session once from `/assets/models/onnx/best.onnx`; runs via `onnxruntime-web`; output shape `[1, N, 6]` (`x, y, x, y, conf, classIdx`); filters at confidence ≥ 0.25.
+- **TFLite path (`local-tflite`):** Dynamically imports `/assets/tflite-wasm/tflite_web_api_client.js`; initializes `TFLiteWebModelRunner` for `/assets/models/tflite/best_float16.tflite`; converts CHW→HWC before upload; model includes internal NMS; output `[1, 300, 6]`; filters at confidence ≥ 0.20; maps `[ymin, xmin, ymax, xmax]` → `[nx1, ny1, nx2, ny2]`.
+
+Returns `InferenceFrame`-shaped `postMessage` correlated by `frameId`.
 
 ### Key Interfaces
 
@@ -105,7 +123,7 @@ interface DetectionResult {
   class: string;              // MPOB class name
   confidence: number;         // [0, 1]
   is_health_alert: boolean;   // true if Abnormal | Empty_Bunch
-  box: [x1, y1, x2, y2];     // pixel coords
+  box: [x1, y1, x2, y2];     // absolute pixel coords
   norm_box?: [nx1, ny1, nx2, ny2]; // normalized [0,1] coords
 }
 
@@ -126,21 +144,21 @@ interface InferenceFrame {
 
 | Alias | Path | Contents |
 |---|---|---|
-| `angularlib/*` | `src/dependencies/angularlib/` | UI components, auth, forms, chat |
-| `dp-ui/*` | `src/dependencies/dp-ui/` | Custom Material extensions, `NgxSocketService` |
-| `fis/*` | `src/dependencies/fis/` | Domain modules (leave, tender, approval) |
+| `angularlib/*` | `src/dependencies/angularlib/` | UI components (forms, dialogs, chat), auth, `UIState`, `ChatState` |
+| `dp-ui/*` | `src/dependencies/dp-ui/` | `DpService`, `NgxSocketService`, `DPState`, FIS message models |
+| `fis/*` | `src/dependencies/fis/` | Domain modules (leave, tender, approval), `MetadataState` |
 | `fis-commons/*` | `src/dependencies/fis-commons/` | CDN-hosted shared utilities |
 
 ### Models & WASM Assets
 
-- `src/assets/models/onnx/best.onnx` — YOLOv8 ONNX model for browser WASM inference
-- `src/assets/models/tflite/best_float16.tflite` / `best_float32.tflite` — TFLite variants
-- `src/assets/wasm/` — ONNX Runtime JS WASM bundles
-- `src/assets/tflite-wasm/` — TensorFlow Lite WASM runtime
+- `src/assets/models/onnx/best.onnx` — YOLOv8 ONNX model for browser ONNX WASM inference
+- `src/assets/models/tflite/best_float16.tflite` — TFLite model (loaded by worker in `local-tflite` mode)
+- `src/assets/wasm/` — ONNX Runtime Web JS + WASM bundles
+- `src/assets/tflite-wasm/` — TensorFlow Lite WASM runtime (`tflite_web_api_client.js` + WASM)
 
 ### Multi-Build Strategy
 
-`angular.json` defines four production configurations that swap assets and menu via `fileReplacements`:
+`angular.json` defines four production configurations that swap menu and assets via `fileReplacements`:
 
 | Config | Output | Swaps |
 |---|---|---|
@@ -153,8 +171,8 @@ Development build uses a mock socket service (`dp.service.t.ts`) for offline dev
 
 ### PWA
 
-Service worker registered immediately (`registerImmediately`) via `ngsw-config.json`. PWA install prompt handled in `AppComponent`. Three language packs: `en_US`, `ms_MY`, `zh_Hans`.
+Service worker registered immediately (`registerImmediately`) via `ngsw-config.json`. PWA install prompt handled in `AppComponent` (mobile platforms only). Three language packs: `en_US`, `ms_MY`, `zh_Hans`.
 
 ### CORS / Connection
 
-Frontend connects to NestJS backend at `http://localhost:3000` (configurable via `config.json`). Backend CORS whitelist is hardcoded in `server-desktop/src/main.ts` — add new device IPs there.
+Frontend connects to NestJS backend at `https://localhost:3000` (configurable via `src/config/config.json`). Backend CORS whitelist is hardcoded in `server-desktop/src/main.ts` and `server-android/src/main.ts` — add new device IPs there.

Rozdielové dáta súboru neboli zobrazené, pretože súbor je príliš veľký
+ 3 - 3
src/assets/tflite-wasm/tflite_web_api_client.js


+ 2 - 2
src/config/config.json

@@ -1,8 +1,8 @@
 {
     "connection": {
         "uacp": "https://localhost:3000",
-        "uacp_ws": "wss://localhost:3000/",
-        "uacpEmulation": "off",
+        "uacp_ws": "wss://localhost:3000",
+        "uacpEmulation": "on",
         "auth": {
             "google": "https://api.swopt.com/auth/google"
         }

+ 76 - 55
src/src.palm.vision/analyzer/analyzer.component.html

@@ -119,69 +119,90 @@
     </div>
   }
 
-  <!-- Canvas bounding box overlay (always in DOM so ViewChild resolves) -->
-  <div class="canvas-container" [class.has-content]="!!currentFrame">
-    <canvas #resultCanvas class="result-canvas"></canvas>
-  </div>
+  <!-- Studio workspace: vertical stack -->
+  <div class="studio-workspace">
+
+    <!-- Canvas bounding box overlay (always in DOM so ViewChild resolves) -->
+    <div class="canvas-container" [class.has-content]="!!currentFrame">
+      <canvas #resultCanvas class="result-canvas"></canvas>
+    </div>
 
-  <!-- Detection results -->
-  @if (currentFrame) {
-    <div class="results-panel">
-
-      <div class="results-header">
-        <span class="results-title">
-          Batch Result &mdash;
-          <strong>{{ currentFrame.total_count }}</strong> detection(s)
-        </span>
-        <span class="timing-info">
-          Inference: {{ currentFrame.inference_ms | number:'1.0-0' }}&nbsp;ms
-          &nbsp;|&nbsp;
-          Processing: {{ currentFrame.processing_ms | number:'1.0-0' }}&nbsp;ms
-        </span>
+    <!-- Batch carousel (visible when batch has multiple frames) -->
+    @if (batchFrames.length > 1) {
+      <div class="carousel-container">
+        <button class="carousel-nav prev" (click)="prevFrame()">&#9664;</button>
+        <div class="carousel-track">
+          @for (frame of batchFrames; track frame.frameId; let i = $index) {
+            <div class="carousel-card" [class.active]="i === selectedFrameIndex" (click)="selectFrame(i)">
+              <span class="card-index">#{{ i + 1 }}</span>
+              <span class="card-meta">{{ frame.detections.length }} det.</span>
+            </div>
+          }
+        </div>
+        <button class="carousel-nav next" (click)="nextFrame()">&#9654;</button>
       </div>
+    }
+
+    <!-- Detection results -->
+    @if (currentFrame) {
+      <div class="results-panel">
+
+        <div class="results-header">
+          <span class="results-title">
+            Batch Result &mdash;
+            <strong>{{ currentFrame.total_count }}</strong> detection(s)
+          </span>
+          <span class="timing-info">
+            Inference: {{ currentFrame.inference_ms | number:'1.0-0' }}&nbsp;ms
+            &nbsp;|&nbsp;
+            Processing: {{ currentFrame.processing_ms | number:'1.0-0' }}&nbsp;ms
+          </span>
+        </div>
 
-      <div class="results-body">
-        <div class="detections-col">
-
-          @if (currentFrame.detections.length) {
-            @for (det of currentFrame.detections; track det.bunch_id) {
-              <mat-card class="detection-card" [class.health-alert]="det.is_health_alert">
-                <mat-card-content>
-                  <span
-                    class="grade-dot"
-                    [ngStyle]="{ background: gradeColor(det.class) }"
-                  ></span>
-                  <span class="grade-label">{{ det.class }}</span>
-                  <span class="confidence-value">{{ confidencePercent(det.confidence) }}</span>
-                  @if (det.is_health_alert) {
-                    <mat-icon class="alert-icon" color="warn">warning</mat-icon>
-                  }
-                </mat-card-content>
-              </mat-card>
+        <div class="results-body">
+          <div class="detections-col">
+
+            @if (currentFrame.detections.length) {
+              @for (det of currentFrame.detections; track det.bunch_id) {
+                <mat-card class="detection-card" [class.health-alert]="det.is_health_alert">
+                  <mat-card-content>
+                    <span
+                      class="grade-dot"
+                      [ngStyle]="{ background: gradeColor(det.class) }"
+                    ></span>
+                    <span class="grade-label">{{ det.class }}</span>
+                    <span class="confidence-value">{{ confidencePercent(det.confidence) }}</span>
+                    @if (det.is_health_alert) {
+                      <mat-icon class="alert-icon" color="warn">warning</mat-icon>
+                    }
+                  </mat-card-content>
+                </mat-card>
+              }
+            } @else {
+              <p class="no-detections">No detections in this frame.</p>
             }
-          } @else {
-            <p class="no-detections">No detections in this frame.</p>
-          }
 
-          @if (currentFrame.total_count > 0) {
-            <div class="summary-block">
-              <h4 class="summary-title">Industrial Summary</h4>
-              <div class="summary-chips">
-                @for (entry of currentFrame.industrial_summary | keyvalue; track entry.key) {
-                  <span
-                    class="summary-chip"
-                    [ngStyle]="{ 'border-color': gradeColor(entry.key) }"
-                  >
-                    {{ entry.key }}: <strong>{{ entry.value }}</strong>
-                  </span>
-                }
+            @if (currentFrame.total_count > 0) {
+              <div class="summary-block">
+                <h4 class="summary-title">Industrial Summary</h4>
+                <div class="summary-chips">
+                  @for (entry of currentFrame.industrial_summary | keyvalue; track entry.key) {
+                    <span
+                      class="summary-chip"
+                      [ngStyle]="{ 'border-color': gradeColor(entry.key) }"
+                    >
+                      {{ entry.key }}: <strong>{{ entry.value }}</strong>
+                    </span>
+                  }
+                </div>
               </div>
-            </div>
-          }
+            }
 
+          </div>
         </div>
       </div>
-    </div>
-  }
+    }
+
+  </div>
 
 </div>

+ 105 - 6
src/src.palm.vision/analyzer/analyzer.component.scss

@@ -204,20 +204,119 @@
 // ── Canvas bounding box overlay ──────────────────────────────────────────────
 
 .canvas-container {
-  display: none;
-  justify-content: center;
   border-radius: 12px;
   overflow: hidden;
-  background: #111;
+  display: none;
 
   &.has-content {
-    display: flex;
+    display: inline-flex;
+    justify-content: center;
+    align-items: center;
+    width: auto;
+    max-width: 100%;
+    background-color: transparent;
   }
 
   .result-canvas {
-    max-width: 100%;
-    height: auto;
     display: block;
+    width: 100%;
+    height: auto;
+    max-height: 70vh;
+  }
+}
+
+// ── Studio workspace vertical stack ──────────────────────────────────────────
+
+.studio-workspace {
+  display: flex;
+  flex-direction: column;
+  gap: 1rem;
+}
+
+// ── Batch carousel ────────────────────────────────────────────────────────────
+
+.carousel-container {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  width: 100%;
+}
+
+.carousel-nav {
+  flex-shrink: 0;
+  width: 32px;
+  height: 32px;
+  border: 1.5px solid #a5d6a7;
+  border-radius: 50%;
+  background: #f9fbe7;
+  color: #2e7d32;
+  font-size: 0.8rem;
+  cursor: pointer;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  transition: background 0.15s, border-color 0.15s;
+
+  &:hover {
+    background: #e8f5e9;
+    border-color: #2e7d32;
+  }
+}
+
+.carousel-track {
+  flex: 1;
+  display: flex;
+  gap: 0.5rem;
+  overflow-x: auto;
+  padding-bottom: 4px;
+  scrollbar-width: thin;
+  scrollbar-color: #a5d6a7 transparent;
+
+  &::-webkit-scrollbar {
+    height: 4px;
+  }
+
+  &::-webkit-scrollbar-thumb {
+    background: #a5d6a7;
+    border-radius: 2px;
+  }
+}
+
+.carousel-card {
+  flex-shrink: 0;
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  justify-content: center;
+  gap: 2px;
+  padding: 0.35rem 0.6rem;
+  border-radius: 8px;
+  border: 1.5px solid #c8e6c9;
+  background: #f9fbe7;
+  cursor: pointer;
+  transition: border-color 0.15s, background 0.15s;
+  min-width: 52px;
+
+  &:hover {
+    border-color: #81c784;
+    background: #e8f5e9;
+  }
+
+  &.active {
+    border-color: #2e7d32;
+    background: #e8f5e9;
+  }
+
+  .card-index {
+    font-size: 0.82rem;
+    font-weight: 700;
+    color: #2e7d32;
+  }
+
+  .card-meta {
+    font-size: 0.65rem;
+    color: #607d8b;
+    white-space: nowrap;
   }
 }
 

+ 44 - 7
src/src.palm.vision/analyzer/analyzer.component.ts

@@ -55,6 +55,8 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
   isDragOver = false;
   isCameraActive = false;
   currentFrame: InferenceFrame | null = null;
+  batchFrames: InferenceFrame[] = [];
+  selectedFrameIndex = 0;
 
   readonly engineOptions: { value: EngineMode; label: string; icon: string }[] = [
     { value: 'local-onnx', label: 'Local — ONNX WASM', icon: 'memory' },
@@ -64,6 +66,7 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
 
   private mediaStream: MediaStream | null = null;
   private inferenceSub!: Subscription;
+  private batchSub!: Subscription;
 
   constructor(private store: Store) {}
 
@@ -76,6 +79,12 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
           this.renderPredictionsWithBoxes(frame);
         }
       });
+
+    this.batchSub = (this.store.select((state: any) => state.visionState?.batchFrames ?? []) as Observable<InferenceFrame[]>)
+      .subscribe(frames => {
+        this.batchFrames = frames;
+        this.selectedFrameIndex = 0;
+      });
   }
 
   ngAfterViewInit(): void {
@@ -88,6 +97,7 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
   ngOnDestroy(): void {
     this.stopCamera();
     this.inferenceSub?.unsubscribe();
+    this.batchSub?.unsubscribe();
   }
 
   // ── Input mode ────────────────────────────────────────────────────────────
@@ -171,10 +181,15 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
 
       const img = new Image();
       img.onload = () => {
-        canvas.width = img.naturalWidth;
-        canvas.height = img.naturalHeight;
+        const container = this.resultCanvasRef.nativeElement.parentElement;
+        const targetWidth = container?.clientWidth || img.naturalWidth;
+        const targetHeight = container?.clientHeight || img.naturalHeight;
+        const scale = Math.min(targetWidth / img.naturalWidth, targetHeight / img.naturalHeight);
+
+        canvas.width = img.naturalWidth * scale;
+        canvas.height = img.naturalHeight * scale;
         const ctx = canvas.getContext('2d')!;
-        ctx.drawImage(img, 0, 0);
+        ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
 
         for (const det of frame.detections) {
           let x: number, y: number, w: number, h: number;
@@ -186,10 +201,10 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
             h = (ny2 - ny1) * canvas.height;
           } else if (det.box) {
             const [x1, y1, x2, y2] = det.box;
-            x = x1;
-            y = y1;
-            w = x2 - x1;
-            h = y2 - y1;
+            x = x1 * scale;
+            y = y1 * scale;
+            w = (x2 - x1) * scale;
+            h = (y2 - y1) * scale;
           } else {
             continue;
           }
@@ -230,6 +245,28 @@ export class AnalyzerComponent implements OnInit, AfterViewInit, OnDestroy {
     return palette[cls] ?? '#757575';
   }
 
+  selectFrame(index: number): void {
+    this.selectedFrameIndex = index;
+    this.currentFrame = this.batchFrames[index];
+    if (this.isViewInitialized) {
+      this.renderPredictionsWithBoxes(this.batchFrames[index]);
+    }
+  }
+
+  prevFrame(): void {
+    const next = this.selectedFrameIndex > 0
+      ? this.selectedFrameIndex - 1
+      : this.batchFrames.length - 1;
+    this.selectFrame(next);
+  }
+
+  nextFrame(): void {
+    const next = this.selectedFrameIndex < this.batchFrames.length - 1
+      ? this.selectedFrameIndex + 1
+      : 0;
+    this.selectFrame(next);
+  }
+
   private submit(files: File[]): void {
     this.store.dispatch(new SubmitBatchAnalysis({ files, mode: this.mode }));
   }

+ 23 - 7
src/src.palm.vision/history/history.component.ts

@@ -115,12 +115,24 @@ export class HistoryComponent implements OnInit, AfterViewInit, OnDestroy {
       ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
 
       for (const det of (item.detections ?? [])) {
-        if (!det.norm_box) continue;
-        const [nx1, ny1, nx2, ny2] = det.norm_box;
-        const x = nx1 * canvas.width;
-        const y = ny1 * canvas.height;
-        const w = (nx2 - nx1) * canvas.width;
-        const h = (ny2 - ny1) * canvas.height;
+        let x: number, y: number, w: number, h: number;
+        if (det.norm_box) {
+          const [nx1, ny1, nx2, ny2] = det.norm_box;
+          x = nx1 * canvas.width;
+          y = ny1 * canvas.height;
+          w = (nx2 - nx1) * canvas.width;
+          h = (ny2 - ny1) * canvas.height;
+        } else if (det.box) {
+          const scaleX = canvas.width / img.naturalWidth;
+          const scaleY = canvas.height / img.naturalHeight;
+          const [x1, y1, x2, y2] = det.box;
+          x = x1 * scaleX;
+          y = y1 * scaleY;
+          w = (x2 - x1) * scaleX;
+          h = (y2 - y1) * scaleY;
+        } else {
+          continue;
+        }
         const color = this.gradeColor(det.class);
 
         ctx.strokeStyle = color;
@@ -156,8 +168,12 @@ export class HistoryComponent implements OnInit, AfterViewInit, OnDestroy {
   }
 
   private buildGroups(items: any[], expandedIds: string[]): BatchGroup[] {
+    if (!items || !Array.isArray(items)) {
+      return [];
+    }
+
     const groupMap = new Map<string, any[]>();
-    for (const item of items) {
+    for (const item of (items ?? [])) {
       const bid = item.batch_id ?? 'unknown';
       if (!groupMap.has(bid)) groupMap.set(bid, []);
       groupMap.get(bid)!.push(item);

+ 36 - 6
src/src.palm.vision/services/remote-inference.service.ts

@@ -1,6 +1,6 @@
 import { HttpClient } from '@angular/common/http';
 import { Injectable, OnDestroy } from '@angular/core';
-import { Observable, Subject, take } from 'rxjs';
+import { map, Observable, Subject, take } from 'rxjs';
 import { DpService } from 'dp-ui/dp.service';
 import { FisAppMessage, MessageHeader, AppMessageType } from 'dp-ui/fisappmessage/apprequestmessagetype';
 import { InferenceFrame, DetectionResult, MPOB_CLASSES, HEALTH_ALERT_CLASSES } from './inference.service';
@@ -55,7 +55,24 @@ export class RemoteInferenceService implements OnDestroy {
   }
 
   getHistory(): Observable<any[]> {
-    return this.send<any[]>('History', 'getAll', undefined);
+    return this.send<any>('History', 'getAll', undefined).pipe(
+      map(body => {
+        console.log('[Vault Transport Debug] Raw History Network Payload:', body);
+
+        if (Array.isArray(body)) {
+          return body;
+        }
+
+        if (body) {
+          const extractedRecords = body.records || body.data || body.items || body.history;
+          if (Array.isArray(extractedRecords)) {
+            return extractedRecords;
+          }
+        }
+
+        return [];
+      })
+    );
   }
 
   deleteRecord(archiveId: string): Observable<{ deleted: boolean }> {
@@ -71,7 +88,16 @@ export class RemoteInferenceService implements OnDestroy {
   }
 
   saveExternalResult(payload: EdgeResultPayload): Observable<any> {
-    return this.send('PalmHistory', 'SaveExternalResult', payload);
+    const adjustedPayload: any = {
+      ...payload,
+      frame: (payload as any).frame || (payload as any).imageDataUrl || '',
+    };
+
+    if (adjustedPayload.imageDataUrl) {
+      delete adjustedPayload.imageDataUrl;
+    }
+
+    return this.send('PalmHistory', 'SaveExternalResult', adjustedPayload);
   }
 
   ngOnDestroy(): void {
@@ -101,6 +127,12 @@ export class RemoteInferenceService implements OnDestroy {
       // Direct call routing through the shared enterprise stream engine
       this.dpService.stream(message).subscribe({
         next: (res: any) => {
+          // Gracefully intercept and isolate system finalization frames before parsing
+          if (res && res.complete === true && !res.message) {
+            observer.complete();
+            return;
+          }
+
           // Extract body mapping parameters directly from enterprise results packets
           const body = typeof res === 'string' ? JSON.parse(res) : (res?.message ? JSON.parse(res.message) : res);
           if (body?.error) {
@@ -132,9 +164,7 @@ export class RemoteInferenceService implements OnDestroy {
     return {
       frameId: raw?.archive_id ?? crypto.randomUUID(),
       batchId,
-      imageDataUrl: raw?.image_data
-        ? `data:image/jpeg;base64,${raw.image_data}`
-        : '',
+      imageDataUrl: raw?.image_data ?? raw?.imageDataUrl ?? '',
       detections,
       inference_ms: raw?.inference_ms ?? 0,
       processing_ms: raw?.processing_ms ?? 0,

+ 19 - 5
src/src.palm.vision/store/vision.state.ts

@@ -17,6 +17,8 @@ export interface VisionStateModel {
   loading: boolean;
   expandedBatchIds: string[];
   currentInference: any | null;
+  batchFrames: any[];
+  selectedFrameIndex: number | null;
 }
 
 const defaults: VisionStateModel = {
@@ -24,6 +26,8 @@ const defaults: VisionStateModel = {
   loading: false,
   expandedBatchIds: [],
   currentInference: null,
+  batchFrames: [],
+  selectedFrameIndex: null,
 };
 
 @State<VisionStateModel>({
@@ -104,7 +108,14 @@ export class VisionState implements NgxsOnInit {
     return merge(...streams).pipe(
       tap(frame => ctx.patchState({ currentInference: frame })),
       toArray(),
-      tap(() => ctx.patchState({ loading: false })),
+      tap(frames => {
+        ctx.patchState({
+          batchFrames: frames,
+          selectedFrameIndex: frames.length > 0 ? 0 : null,
+          loading: false,
+        });
+        ctx.dispatch(new LoadHistory());
+      }),
     );
   }
 
@@ -140,9 +151,12 @@ export class VisionState implements NgxsOnInit {
           tap(res => {
             const current = ctx.getState().items;
             ctx.patchState({
-              items: current.map((i: any) =>
-                i.archive_id === res.archiveId ? { ...i, imageDataUrl: res.image_data } : i,
-              ),
+              items: current.map((i: any) => {
+                // Resolve the identifier matching key from snake_case or camelCase network variants
+                const r = res as any;
+                const inboundId = r?.archive_id || r?.archiveId;
+                return i.archive_id === inboundId ? { ...i, imageDataUrl: (r?.image_data || r?.imageDataUrl) } : i;
+              }),
             });
           }),
         ),
@@ -154,7 +168,7 @@ export class VisionState implements NgxsOnInit {
   loadHistory(ctx: StateContext<VisionStateModel>): Observable<void> {
     ctx.patchState({ loading: true });
     return this.remoteInferenceService.getHistory().pipe(
-      timeout(5000),
+      timeout(30000), // Extended to 30s to shield slow edge database sweeps safely
       tap(items => ctx.patchState({ items, loading: false })),
       catchError(err => {
         console.warn('⚠️ [Vault State] Edge network connection lost or timed out:', err.message || err);

+ 143 - 8
src/src.palm.vision/workers/inference.worker.ts

@@ -5,7 +5,7 @@ ort.env.wasm.wasmPaths = '/assets/wasm/';
 ort.env.wasm.numThreads = 1;
 
 let onnxSession: ort.InferenceSession | null = null;
-let tfliteSession: any = null; // Target holder for parallel TFLite engine weights loops
+let tfliteSession: any = null;
 
 async function initOnnxSession(): Promise<ort.InferenceSession> {
   if (onnxSession) return onnxSession;
@@ -15,6 +15,39 @@ async function initOnnxSession(): Promise<ort.InferenceSession> {
   return onnxSession;
 }
 
+async function initTfliteSession(): Promise<any> {
+  if (tfliteSession) return tfliteSession;
+
+  if (!(globalThis as any).Module) {
+    (globalThis as any).Module = {
+      locateFile: (path: string) => {
+        // Direct the internal loader to fetch the correct compiled _cc binaries explicitly
+        return `/assets/tflite-wasm/${path}`;
+      },
+    };
+  }
+
+  if (!(globalThis as any).tfweb) {
+    // Mask the path in a runtime variable to hide it from esbuild static analysis
+    const runtimeAssetPath = '/assets/tflite-wasm/tflite_web_api_client.js';
+    await import(/* @vite-ignore */ runtimeAssetPath as any);
+  }
+
+  const tfwebGlobal = (globalThis as any).tfweb;
+  if (!tfwebGlobal || !tfwebGlobal.TFLiteWebModelRunner) {
+    throw new Error('[TFLite] TFLiteWebModelRunner constructor not found on global scope');
+  }
+
+  if (tfwebGlobal.tflite_web_api && typeof tfwebGlobal.tflite_web_api.setWasmPath === 'function') {
+    tfwebGlobal.tflite_web_api.setWasmPath('/assets/tflite-wasm/');
+  }
+
+  tfliteSession = await tfwebGlobal.TFLiteWebModelRunner.create(
+    '/assets/models/tflite/best_float16.tflite',
+  );
+  return tfliteSession;
+}
+
 const MPOB_CLASSES = ['Empty_Bunch', 'Underripe', 'Abnormal', 'Ripe', 'Unripe', 'Overripe'];
 const CONF_THRESHOLD = 0.25;
 
@@ -30,12 +63,114 @@ addEventListener('message', async ({ data }) => {
 
     // ── Engine Routing Branch Matrix ──────────────────────────────────────────
     if (mode === 'local-tflite') {
-      // ⚠️ TFLite WASM Pipeline Fallback Placeholder Loop
-      // Explicitly capture TFLite routing and return structural data without freezing threads
-      console.warn('[Worker] TFLite WASM engine branch hit — utilizing float metrics');
+      const runner = await initTfliteSession();
+
+      // Reconstruct typed view over the transferred ArrayBuffer — prevents undefined/NaN indexing
+      const rawPlanarData = new Float32Array(tensor);
+      const pixelCount = 640 * 640;
+
+      // Build a clean intermediate HWC buffer before touching WASM memory
+      const hwcData = new Float32Array(pixelCount * 3);
+      for (let i = 0; i < pixelCount; i++) {
+        hwcData[i * 3]     = rawPlanarData[i];                  // R
+        hwcData[i * 3 + 1] = rawPlanarData[pixelCount + i];     // G
+        hwcData[i * 3 + 2] = rawPlanarData[2 * pixelCount + i]; // B
+      }
+
+      const inputTensor = runner.getInputs()[0];
+      if (!inputTensor) {
+        throw new Error('[TFLite] Missing input tensor description definition');
+      }
+
+      // Resolve the true underlying memory buffer — data may be a function closure over WASM heap
+      const inputBuffer = typeof inputTensor.data === 'function'
+        ? (inputTensor.data as any)()
+        : inputTensor.data;
+
+      if (inputBuffer) {
+        if (typeof inputBuffer.set === 'function') {
+          inputBuffer.set(hwcData);
+        } else {
+          for (let i = 0; i < hwcData.length; i++) {
+            inputBuffer[i] = hwcData[i];
+          }
+        }
+      } else {
+        throw new Error('[TFLite] Failed to initialize a valid WebAssembly memory view for input data');
+      }
+
+      runner.infer();
+
+      const outputs = runner.getOutputs();
+      console.log('[TFLite Debug] Input Matrix Shape:', runner.getInputs()[0].shape);
+
+      const outputTensor = outputs[0];
+      let outputShape: number[] = [];
 
-      // Temporary structural pass-through until local .tflite weights parsing loop is active
-      detections = [];
+      if (typeof outputTensor.shape === 'string') {
+        outputShape = (outputTensor.shape as any).split(',').map((v: string) => parseInt(v, 10));
+      } else if (Array.isArray(outputTensor.shape)) {
+        outputShape = outputTensor.shape;
+      } else if (outputTensor.shape && typeof (outputTensor.shape as any).toArray === 'function') {
+        outputShape = (outputTensor.shape as any).toArray();
+      } else {
+        outputShape = [1, 300, 6];
+      }
+      console.log('[TFLite Debug] Final Parsed Dimensions Array:', outputShape);
+
+      // Native engine accessor API — bypasses function-closure indirection over WASM heap
+      let rawValues: any = null;
+      if (typeof (runner as any).getOutputTensorData === 'function') {
+        rawValues = (runner as any).getOutputTensorData(0);
+      } else if (typeof (runner as any).getOutputData === 'function') {
+        rawValues = (runner as any).getOutputData(0);
+      } else {
+        rawValues = outputTensor && typeof outputTensor.data === 'function'
+          ? (outputTensor.data as any)()
+          : (outputTensor ? outputTensor.data : null);
+      }
+      console.log('[TFLite Debug] Real Array Check:', Array.isArray(rawValues), rawValues?.constructor?.name);
+      const debugSlice = rawValues
+        ? Array.from(rawValues.subarray ? rawValues.subarray(0, 12) : (rawValues.slice ? rawValues.slice(0, 12) : rawValues))
+        : 'null';
+      console.log('[TFLite Debug] Real Slice:', debugSlice);
+
+      // Model has internal NMS — output is [1, 300, 6] post-NMS candidates
+      // Bypass manual CONF_THRESHOLD: model already filtered; skip only zero-confidence empty slots
+      const numCandidates = outputShape[1] || 300;
+      const stride = outputShape[2] || 6;
+
+      for (let i = 0; i < numCandidates; i++) {
+        const offset = i * stride;
+        const confidence = rawValues[offset + 4];
+
+        // Relax the gateway filter to capture both live detections explicitly
+        if (isNaN(confidence) || confidence < 0.20) {
+          continue;
+        }
+
+        // Extract the true class index predicted by the WebAssembly model graph
+        const classIdx = Math.round(rawValues[offset + 5]);
+        const className = MPOB_CLASSES[classIdx] || 'Unknown';
+
+        // Map TFLite NMS format [ymin, xmin, ymax, xmax] to standard UI layout [x1, y1, x2, y2]
+        const ny1 = parseFloat(rawValues[offset + 0].toFixed(6)); // ymin
+        const nx1 = parseFloat(rawValues[offset + 1].toFixed(6)); // xmin
+        const ny2 = parseFloat(rawValues[offset + 2].toFixed(6)); // ymax
+        const nx2 = parseFloat(rawValues[offset + 3].toFixed(6)); // xmax
+
+        industrialSummary[className] = (industrialSummary[className] ?? 0) + 1;
+
+        console.log(`[TFLite Loop Success] Pushing candidate ${i}, Conf: ${confidence.toFixed(4)}, Class: ${classIdx}`);
+        detections.push({
+          bunch_id: detections.length + 1,
+          class: className,
+          confidence: parseFloat(confidence.toFixed(4)),
+          is_health_alert: ['Abnormal', 'Empty_Bunch'].includes(className),
+          norm_box: [nx1, ny1, nx2, ny2],
+          box: [nx1 * 640, ny1 * 640, nx2 * 640, ny2 * 640],
+        });
+      }
     } else {
       // Default to standard local ONNX execution stream
       const ortSession = await initOnnxSession();
@@ -83,7 +218,7 @@ addEventListener('message', async ({ data }) => {
       imageDataUrl,
       detections,
       inference_ms: parseFloat(inferenceMs.toFixed(2)),
-      processing_ms: parseFloat((performance.now() - processingStart).toFixed(2)),
+      processing_ms: parseFloat(Math.max(0, performance.now() - processingStart).toFixed(2)),
       total_count: detections.length,
       industrial_summary: industrialSummary,
       source: 'wasm-local',
@@ -97,7 +232,7 @@ addEventListener('message', async ({ data }) => {
       imageDataUrl,
       detections: [],
       inference_ms: 0,
-      processing_ms: parseFloat((performance.now() - processingStart).toFixed(2)),
+      processing_ms: parseFloat(Math.max(0, performance.now() - processingStart).toFixed(2)),
       total_count: 0,
       industrial_summary: {},
       source: 'wasm-local',

Niektoré súbory nie sú zobrazené, pretože je v týchto rozdielových dátach zmenené mnoho súborov