updated readme

README.md

@@ -1,255 +1,76 @@
 # 🌴 Palm Oil FFB Management System (YOLO26)
+> **⚠️ PROTOTYPE / PROOF OF CONCEPT (PoC)** > This system is developed strictly for demonstration purposes to potential clients and is **not intended for production use** in its current state.
 
-A production-ready AI system for detecting the ripeness of Palm Oil Fresh Fruit Bunches (FFB). Built on a custom-trained **YOLO26** model (YOLOv8 architecture fork) with a **dual-engine inference backend** (ONNX + PyTorch), a FastAPI server, and a full-featured Streamlit dashboard. The entire backend is architected with **Domain-Driven Design (DDD)** for maximum scalability and n8n workflow integration.
+A high-fidelity AI prototype for detecting the ripeness of Palm Oil Fresh Fruit Bunches (FFB) according to MPOB standards. Built on a custom-trained **YOLO26** model (utilizing the latest high-performance architecture via `ultralytics`) with a **triple-engine inference backend** and a specialized 5-tab operations dashboard.
 
+LINK for GOOGLE DOC: https://docs.google.com/document/d/1bbdGuLCGHdnHj_iCQIreh9DlYwu12PDUEPpA2eCsR-M/edit?usp=sharing
 ---
 
 ## 🚀 Project Overview
 
 | Component | Technology | Purpose |
 | :--- | :--- | :--- |
-| **Vision Engine** | YOLO26 (Custom-trained on MPOB-standard datasets) | FFB Ripeness Detection |
-| **ONNX Runtime** | `onnxruntime` + `best.onnx` | Zero-latency, NMS-Free edge inference (~39ms) |
-| **PyTorch Runtime** | `ultralytics` + `best.pt` | High-resolution auditing inference |
-| **Benchmark Engine** | YOLOv8-Sawit (`sawit_tbs.pt`) | Third-party model comparison |
-| **Inference Server** | FastAPI (Python) | REST API for n8n & mobile integration |
-| **Visual Fingerprinting** | Vertex AI Multimodal Embedding (`multimodalembedding@001`) | 1408-D vector generation |
-| **Cloud Archival** | MongoDB Atlas Vector Search | Similarity-based semantic recall |
-| **Local History** | SQLite (`palm_history.db`) | Offline audit log, zero cloud dependency |
-| **Demo Dashboard** | Streamlit (`demo_app.py`) | 5-tab production operations UI |
+| **Primary Engine** | YOLO26 (Latest SOTA Backbone) | High-accuracy FFB Ripeness Detection. |
+| **ONNX Runtime** | `onnxruntime` + `best.onnx` | Ultra-fast edge inference simulation (~39ms). |
+| **PyTorch Runtime** | `ultralytics` + `best.pt` | High-resolution auditing and verification. |
+| **Benchmark Engine** | YOLOv8-Sawit (`sawit_tbs.pt`) | Competitive baseline comparison. |
+| **Inference Server** | FastAPI (Python) | REST API for multi-client orchestration. |
+| **Local Audit Log** | SQLite (`palm_history.db`) | **Primary** offline-first data persistence. |
+| **Operations UI** | Streamlit (`demo_app.py`) | 5-tab industrial management dashboard. |
 
 ---
 
-## 🛠 Prerequisites
+## 🛠️ Triple-Engine Inference Logic
+The system demonstrates three distinct AI pathways to showcase deployment flexibility:
 
-- Python 3.10+
-- An NVIDIA GPU (recommended, but not required — CPU inference is supported)
-- n8n (Desktop or Self-hosted) for workflow automation
-- MongoDB Atlas Account *(optional — required only for cloud archival & semantic search)*
-- Google Cloud Platform with Vertex AI API enabled *(optional — required only for vectorization)*
+1.  **YOLO26 (ONNX)**: Optimized for zero-latency scenarios. It utilizes normalized coordinates ($0.0$ to $1.0$) mapped back to original resolution via a custom scaling handler.
+2.  **YOLO26 (PyTorch)**: The native "Ground Truth" engine. It provides absolute pixel coordinates and serves as the benchmark for high-resolution audits.
+3.  **YOLOv8-Sawit (Benchmark)**: A third-party model implementation. It allows clients to see how the custom YOLO26 architecture performs against standard industry baselines.
 
 ---
 
-## 📦 Setup Instructions
+## 🖥️ Demonstration Features (Streamlit)
 
-### 1. Environment Setup
+* **Tab 1: Single Analysis**: Drag-and-drop detection with interactive Plotly overlays and a "Manager's Dashboard" for OER (Oil Extraction Rate) yield-loss insights.
+* **Tab 2: Batch Processing**: Simulates a full harvest day by processing multiple images under a specific metadata context (Estate, Block, Harvester) and generating an **Executive PDF Report**.
+* **Tab 3: Similarity Search (KIV)**: *Keep In View.* Currently staged for future integration. Designed for visual and natural language search via Vertex AI.
+* **Tab 4: History Vault**: A local audit log showing every analysis performed. It displays archived images alongside their raw mathematical tensors.
+* **Tab 5: Batch Reviewer**: An auditor's tool to browse `manifest.json` data contracts and verify resolution-agnostic `norm_box` coordinates.
 
-```powershell
-# Clone and enter the repository
-git clone <your-repo-url>
-cd palm-oil-ai
-
-# Create and activate virtual environment
-python -m venv venv
-.\venv\Scripts\activate
+---
 
-# Install dependencies
-pip install -r requirements.txt
-```
+## 🚧 Status & Future Roadmap (KIV)
+The following features are currently **KIV (Keep In View)** and are disabled in the current prototype due to cloud service constraints:
 
-> **Note:** `onnxruntime` and `fpdf2` are required but not yet in `requirements.txt`. Install manually if needed:
-> ```powershell
-> pip install onnxruntime fpdf2
-> ```
-
-### 2. Dataset & Training
-
-1. Download the dataset from [Roboflow](https://universe.roboflow.com/assignment-vvtq7/oil-palm-ripeness/dataset/5/download/yolov8) or source your own (ensure consistent YOLO `.yaml` structure).
-2. Extract into `/datasets`.
-3. **Train the model:**
-    ```bash
-    python train_palm.py
-    ```
-4. Copy the resulting `best.pt` from `runs/detect/train/weights/` to the project root.
-5. **Export to ONNX** for high-speed inference:
-    ```bash
-    python export_raw_tflite.py  # or use yolo export
-    ```
-    Copy the resulting `best.onnx` to the project root.
-
-### 3. Configuration (`.env`)
-
-Populate your `.env` file. Cloud services (Vertex AI, MongoDB) are **optional** — the system gracefully degrades to local-only mode if they are unavailable.
-
-```env
-# Required for Cloud Archival & Semantic Search
-MONGO_URI=mongodb+srv://<user>:<password>@<cluster>.mongodb.net/
-PROJECT_ID=your-gcp-project-id
-LOCATION=us-central1
-DB_NAME=palm_oil_db
-COLLECTION_NAME=ffb_records
-
-# Path to your GCP Service Account key JSON
-GOOGLE_APPLICATION_CREDENTIALS=gemini-embedding-service-key.json
-```
+* **Cloud Archival**: Automatic vectorization of detections to MongoDB Atlas.
+* **Semantic Search**: Visual similarity and Natural Language search via Vertex AI Multi-modal embeddings.
+* **Vertex AI Integration**: Currently experiencing issues; the system gracefully degrades to **Local-Only Mode** for stable demonstrations.
 
 ---
 
-## 🚦 How to Run
-
-### Start the FastAPI Backend
-
-The API server is the **required** component. The Streamlit dashboard will not function without it.
+## 🔌 Running the Demonstration
 
+### 1. Start the FastAPI Backend
+The API server must be running to provide the NMS-Free inference logic.
 ```powershell
-# Start the FastAPI server (root-level wrapper)
 python main.py
 ```
 
-The server will be available at `http://localhost:8000`. Interactive API docs are at `http://localhost:8000/docs`.
-
-*Alternatively, run as a module: `python -m src.api.main`*
-
-### Start the Streamlit Dashboard
-
-Open a **second terminal** and run:
-
+### 2. Launch the Operations Dashboard
+In a separate terminal, run the Streamlit UI:
 ```powershell
 streamlit run demo_app.py
 ```
 
-The dashboard automatically connects to the backend and will display an error with a retry button if the API is offline.
-
----
-
-## 🔌 API Endpoints
-
-| Endpoint | Method | Description |
-| :--- | :--- | :--- |
-| `/analyze` | `POST` | **Single Analysis**: Runs inference on one image; auto-archives to local SQLite vault. Accepts `model_type` form field (`onnx`, `pytorch`, `yolov8_sawit`). |
-| `/process_batch` | `POST` | **Batch Processor**: Processes multiple images; generates a `manifest.json` data contract in `batch_outputs/`. Accepts `model_type` and `metadata` (JSON string). |
-| `/vectorize_and_store` | `POST` | **Cloud Archival**: Vectorizes a single detection and saves to MongoDB Atlas. Requires active GCP billing. |
-| `/search_hybrid` | `POST` | **Semantic Search**: Visual similarity (upload image) or natural language query via Vertex AI embeddings. |
-| `/get_history` | `GET` | **History Vault**: Returns all records from the local SQLite audit log, ordered by most recent. |
-| `/get_image/{record_id}` | `GET` | **Image Retrieval**: Returns the Base64-encoded image for a specific MongoDB record. |
-| `/get_model_info` | `GET` | Returns the available detection categories and description for the specified `model_type`. |
-| `/get_confidence` | `GET` | Retrieves the current global AI confidence threshold. |
-| `/set_confidence` | `POST` | Updates the AI confidence threshold globally (live, no restart required). |
-
----
-
-## 🖥️ Streamlit Dashboard Tabs
-
-The dashboard (`demo_app.py`) features a **5-tab production operations UI**:
-
-| Tab | Feature | Description |
-| :--- | :--- | :--- |
-| **Single Analysis** | Live Detection | Drag-and-drop a single image for auto-detection. Includes an interactive Plotly overlay viewer, a Manager's Dashboard (metrics), raw tensor inspector, harvest quality pie chart, OER yield-loss insights, cloud archival button, and misclassification flagging. |
-| **Batch Processing** | Bulk Analysis | Upload multiple images and configure production metadata (Estate, Block ID, Harvester ID, Priority) via a modal dialog. Displays a batch quality dashboard (bar chart), annotated evidence gallery, performance timeline (start/end/duration), and generates a **downloadable PDF executive report**. |
-| **Similarity Search** | Semantic Search | Search the MongoDB Atlas vector index by uploading a reference image (visual similarity) or typing a natural language query (text-to-vector). |
-| **History Vault** | Local Audit Log | SQLite-backed audit log of every `/analyze` call. Supports a list view (filterable dataframe) and a "Deep Dive" detail view with interactive Plotly + static annotated image views and the raw mathematical tensor. |
-| **Batch Reviewer** | Manifest Auditor | Browses batches saved in the `batch_outputs/` directory. Loads `manifest.json` data contracts, displays the full batch metadata audit (Job ID, venue, engine, threshold, performance timeline), a quality overview chart, and a per-image inventory with interactive detection overlays and Subscriber Payloads (clean ERP-ready JSON). |
-
-### Sidebar Controls
-
-- **Confidence Threshold**: Live slider (0.1–1.0) that updates the backend globally in real-time.
-- **Model Engine Selector**: Switch between YOLO26 (ONNX), YOLO26 (PyTorch), and YOLOv8-Sawit (Benchmark). Switching engines automatically clears the current analysis canvas.
-- **Model Capabilities Panel**: Dynamically shows the detection categories for the selected engine.
-- **AI Interpretation Guide**: A built-in dialog explaining the raw tensor format, coordinate systems (normalized vs. absolute pixels), and the confidence scoring mechanism.
-
----
-
-## 📦 Batch Output Contract (`manifest.json`)
-
-Each batch job produces a portable data bundle under `batch_outputs/<BATCH_ID>/`:
-
-```
-batch_outputs/
-└── BATCH_<ID>/
-    ├── manifest.json   # The Data Contract
-    └── raw/            # Original uploaded images
-        ├── <uid>_image1.jpg
-        └── <uid>_image2.jpg
-```
-
-The `manifest.json` schema:
-
-```json
-{
-  "job_id": "BATCH_XXXXXXXX",
-  "timestamp": "2026-03-30T...",
-  "source_context": { "estate": "...", "block": "...", "harvester": "...", "priority": "..." },
-  "engine": { "name": "YOLO26", "type": "onnx", "threshold": 0.25 },
-  "performance": { "start_time": "...", "end_time": "...", "duration_seconds": 1.23 },
-  "industrial_summary": { "Ripe": 5, "Unripe": 1, "Underripe": 2, "Abnormal": 0, "Empty_Bunch": 0, "Overripe": 0 },
-  "inventory": [
-    {
-      "image_id": "abc123",
-      "filename": "abc123_image.jpg",
-      "inference_ms": 38.5,
-      "raw_tensor": [...],
-      "detections": [
-        {
-          "bunch_id": 1, "class": "Ripe", "confidence": 0.92,
-          "is_health_alert": false,
-          "box": [x1, y1, x2, y2],
-          "norm_box": [0.1, 0.2, 0.5, 0.8]
-        }
-      ]
-    }
-  ]
-}
-```
-
-> **Note:** `norm_box` stores resolution-agnostic normalized coordinates (0.0–1.0), enabling the Batch Reviewer to re-render detections on any image resolution without data loss.
+### 3. Local Data Contract
+Every batch job generates a `manifest.json` in `batch_outputs/`. This file demonstrates the "Subscriber Payload"—a clean JSON structure ready for hand-off to corporate ERP systems or future vector databases.
 
 ---
 
 ## 🏗️ Architecture (DDD)
+The prototype follows **Domain-Driven Design** to ensure that while this is a PoC, the underlying logic is built on a scalable enterprise foundation:
+* `src/api/`: Model orchestration and API endpoints.
+* `src/application/`: Industrial use cases (Batch, Analysis).
+* `src/domain/`: Core `PalmOilBunch` business entities.
+* `src/infrastructure/`: Repository logic and AI Service wrappers.
 
-```
-palm-oil-ai/
-├── src/
-│   ├── api/
-│   │   └── main.py             # FastAPI routes, ModelManager (ONNX + PyTorch), SQLite auto-archival
-│   ├── application/
-│   │   └── analyze_bunch.py    # Use Cases: AnalyzeBunchUseCase, AnalyzeBatchUseCase, SearchSimilarUseCase
-│   ├── domain/
-│   │   └── models.py           # PalmOilBunch dataclass (core business entity)
-│   └── infrastructure/
-│       ├── repository.py       # MongoPalmOilRepository (Atlas Vector Search, CRUD)
-│       └── vision_service.py   # VertexVisionService (1408-D embeddings, Base64 encoding)
-├── demo_app.py                 # Streamlit 5-tab dashboard
-├── main.py                     # Root-level uvicorn launcher (DDD wrapper)
-├── train_palm.py               # YOLO training script
-├── export_raw_tflite.py        # ONNX/TFLite export utility
-├── best.onnx                   # YOLO26 ONNX weights (primary engine)
-├── best.pt                     # YOLO26 PyTorch weights
-├── sawit_tbs.pt                # YOLOv8-Sawit benchmark weights
-├── palm_history.db             # Local SQLite audit log
-├── batch_outputs/              # Batch job data bundles (manifest + raw images)
-├── history_archive/            # Archived images for History Vault
-├── feedback/                   # Misclassification feedback data (Human-in-the-Loop)
-├── datasets/                   # Labeled training images (Train/Valid/Test)
-├── runs/                       # YOLO training logs and output weights
-├── requirements.txt            # Python dependencies
-├── .env                        # Configuration (secrets, GCP, MongoDB)
-└── README.md                   # You are here
-```
-
-### Detection Classes (MPOB Standard)
-
-| Class | Description | Health Alert |
-| :--- | :--- | :--- |
-| `Ripe` | Prime harvest condition — maximum OER | ❌ |
-| `Underripe` | Harvested before peak — reduces OER | ❌ |
-| `Unripe` | Harvested too early — significant yield loss | ❌ |
-| `Overripe` | Past peak — potential quality degradation | ❌ |
-| `Abnormal` | Disease or structural defect detected | ✅ **CRITICAL** |
-| `Empty_Bunch` | No fruit present — waste indicator | ✅ Warning |
-
----
-
-## 🔑 Key Design Decisions
-
-- **Dual-Engine Inference**: ONNX runtime is the primary engine for its ~39ms NMS-free speed. PyTorch (`.pt`) is retained for high-resolution auditing where standard NMS post-processing is preferred.
-- **Coordinate Normalization**: The batch pipeline stores `norm_box` (0.0–1.0 ratios) alongside absolute pixel `box` coordinates. This makes the data contract resolution-agnostic for downstream ERP or vectorization subscribers.
-- **Graceful Degradation**: MongoDB Atlas and Vertex AI connections are established at startup. If they fail (e.g., no billing, no network), the system logs a warning and continues operating in local-only mode. Only cloud-dependent endpoints return errors.
-- **Human-in-the-Loop**: The "Flag Misclassification" feature in the Single Analysis tab saves flagged images and their detection metadata to a local `feedback/` folder for future model retraining data collection.
-- **SQLite Auto-Archival**: Every call to `/analyze` is automatically logged to `palm_history.db` with the image, detections, engine used, inference/processing latency, and the raw mathematical tensor — enabling a full offline audit trail.
-
----
-
-## 📜 License
-
-This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.