PalmOilRipenessDetection-Python

🌴 Palm Oil FFB Management System (YOLO26)

A production-ready AI system for detecting the ripeness of Palm Oil Fresh Fruit Bunches (FFB). Built on a custom-trained YOLO26 model (YOLOv8 architecture fork) with a dual-engine inference backend (ONNX + PyTorch), a FastAPI server, and a full-featured Streamlit dashboard. The entire backend is architected with Domain-Driven Design (DDD) for maximum scalability and n8n workflow integration.

---

🚀 Project Overview

Component	Technology	Purpose
Vision Engine	YOLO26 (Custom-trained on MPOB-standard datasets)	FFB Ripeness Detection
ONNX Runtime	onnxruntime + best.onnx	Zero-latency, NMS-Free edge inference (~39ms)
PyTorch Runtime	ultralytics + best.pt	High-resolution auditing inference
Benchmark Engine	YOLOv8-Sawit (sawit_tbs.pt)	Third-party model comparison
Inference Server	FastAPI (Python)	REST API for n8n & mobile integration
Visual Fingerprinting	Vertex AI Multimodal Embedding (multimodalembedding@001)	1408-D vector generation
Cloud Archival	MongoDB Atlas Vector Search	Similarity-based semantic recall
Local History	SQLite (palm_history.db)	Offline audit log, zero cloud dependency
Demo Dashboard	Streamlit (demo_app.py)	5-tab production operations UI

---

🛠 Prerequisites

• Python 3.10+
• An NVIDIA GPU (recommended, but not required — CPU inference is supported)
• n8n (Desktop or Self-hosted) for workflow automation
• MongoDB Atlas Account (optional — required only for cloud archival & semantic search)
• Google Cloud Platform with Vertex AI API enabled (optional — required only for vectorization)

---

📦 Setup Instructions

1. Environment Setup

# Clone and enter the repository
git clone <your-repo-url>
cd palm-oil-ai

# Create and activate virtual environment
python -m venv venv
.\venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Note: onnxruntime and fpdf2 are required but not yet in requirements.txt. Install manually if needed:

> pip install onnxruntime fpdf2
> ```

### 2. Dataset & Training

1. Download the dataset from [Roboflow](https://universe.roboflow.com/assignment-vvtq7/oil-palm-ripeness/dataset/5/download/yolov8) or source your own (ensure consistent YOLO `.yaml` structure).
2. Extract into `/datasets`.
3. **Train the model:**
    ```bash
    python train_palm.py
    ```
4. Copy the resulting `best.pt` from `runs/detect/train/weights/` to the project root.
5. **Export to ONNX** for high-speed inference:
    ```bash
    python export_raw_tflite.py  # or use yolo export
    ```
    Copy the resulting `best.onnx` to the project root.

### 3. Configuration (`.env`)

Populate your `.env` file. Cloud services (Vertex AI, MongoDB) are **optional** — the system gracefully degrades to local-only mode if they are unavailable.

```env
# Required for Cloud Archival & Semantic Search
MONGO_URI=mongodb+srv://<user>:<password>@<cluster>.mongodb.net/
PROJECT_ID=your-gcp-project-id
LOCATION=us-central1
DB_NAME=palm_oil_db
COLLECTION_NAME=ffb_records

# Path to your GCP Service Account key JSON
GOOGLE_APPLICATION_CREDENTIALS=gemini-embedding-service-key.json

---

🚦 How to Run

Start the FastAPI Backend

The API server is the required component. The Streamlit dashboard will not function without it.

# Start the FastAPI server (root-level wrapper)
python main.py

The server will be available at http://localhost:8000. Interactive API docs are at http://localhost:8000/docs.

Alternatively, run as a module: python -m src.api.main

Start the Streamlit Dashboard

Open a second terminal and run:

streamlit run demo_app.py

The dashboard automatically connects to the backend and will display an error with a retry button if the API is offline.

---

🔌 API Endpoints

Endpoint	Method	Description
/analyze	POST	Single Analysis: Runs inference on one image; auto-archives to local SQLite vault. Accepts model_type form field (onnx, pytorch, yolov8_sawit).
/process_batch	POST	Batch Processor: Processes multiple images; generates a manifest.json data contract in batch_outputs/. Accepts model_type and metadata (JSON string).
/vectorize_and_store	POST	Cloud Archival: Vectorizes a single detection and saves to MongoDB Atlas. Requires active GCP billing.
/search_hybrid	POST	Semantic Search: Visual similarity (upload image) or natural language query via Vertex AI embeddings.
/get_history	GET	History Vault: Returns all records from the local SQLite audit log, ordered by most recent.
/get_image/{record_id}	GET	Image Retrieval: Returns the Base64-encoded image for a specific MongoDB record.
/get_model_info	GET	Returns the available detection categories and description for the specified model_type.
/get_confidence	GET	Retrieves the current global AI confidence threshold.
/set_confidence	POST	Updates the AI confidence threshold globally (live, no restart required).

---

🖥️ Streamlit Dashboard Tabs

The dashboard (demo_app.py) features a 5-tab production operations UI:

Tab	Feature	Description
Single Analysis	Live Detection	Drag-and-drop a single image for auto-detection. Includes an interactive Plotly overlay viewer, a Manager's Dashboard (metrics), raw tensor inspector, harvest quality pie chart, OER yield-loss insights, cloud archival button, and misclassification flagging.
Batch Processing	Bulk Analysis	Upload multiple images and configure production metadata (Estate, Block ID, Harvester ID, Priority) via a modal dialog. Displays a batch quality dashboard (bar chart), annotated evidence gallery, performance timeline (start/end/duration), and generates a downloadable PDF executive report.
Similarity Search	Semantic Search	Search the MongoDB Atlas vector index by uploading a reference image (visual similarity) or typing a natural language query (text-to-vector).
History Vault	Local Audit Log	SQLite-backed audit log of every /analyze call. Supports a list view (filterable dataframe) and a "Deep Dive" detail view with interactive Plotly + static annotated image views and the raw mathematical tensor.
Batch Reviewer	Manifest Auditor	Browses batches saved in the batch_outputs/ directory. Loads manifest.json data contracts, displays the full batch metadata audit (Job ID, venue, engine, threshold, performance timeline), a quality overview chart, and a per-image inventory with interactive detection overlays and Subscriber Payloads (clean ERP-ready JSON).

Sidebar Controls

• Confidence Threshold: Live slider (0.1–1.0) that updates the backend globally in real-time.
• Model Engine Selector: Switch between YOLO26 (ONNX), YOLO26 (PyTorch), and YOLOv8-Sawit (Benchmark). Switching engines automatically clears the current analysis canvas.
• Model Capabilities Panel: Dynamically shows the detection categories for the selected engine.
• AI Interpretation Guide: A built-in dialog explaining the raw tensor format, coordinate systems (normalized vs. absolute pixels), and the confidence scoring mechanism.

---

📦 Batch Output Contract (manifest.json)

Each batch job produces a portable data bundle under batch_outputs/<BATCH_ID>/:

batch_outputs/
└── BATCH_<ID>/
    ├── manifest.json   # The Data Contract
    └── raw/            # Original uploaded images
        ├── <uid>_image1.jpg
        └── <uid>_image2.jpg

The manifest.json schema:

{
  "job_id": "BATCH_XXXXXXXX",
  "timestamp": "2026-03-30T...",
  "source_context": { "estate": "...", "block": "...", "harvester": "...", "priority": "..." },
  "engine": { "name": "YOLO26", "type": "onnx", "threshold": 0.25 },
  "performance": { "start_time": "...", "end_time": "...", "duration_seconds": 1.23 },
  "industrial_summary": { "Ripe": 5, "Unripe": 1, "Underripe": 2, "Abnormal": 0, "Empty_Bunch": 0, "Overripe": 0 },
  "inventory": [
    {
      "image_id": "abc123",
      "filename": "abc123_image.jpg",
      "inference_ms": 38.5,
      "raw_tensor": [...],
      "detections": [
        {
          "bunch_id": 1, "class": "Ripe", "confidence": 0.92,
          "is_health_alert": false,
          "box": [x1, y1, x2, y2],
          "norm_box": [0.1, 0.2, 0.5, 0.8]
        }
      ]
    }
  ]
}

Note: norm_box stores resolution-agnostic normalized coordinates (0.0–1.0), enabling the Batch Reviewer to re-render detections on any image resolution without data loss.

---

🏗️ Architecture (DDD)

palm-oil-ai/
├── src/
│   ├── api/
│   │   └── main.py             # FastAPI routes, ModelManager (ONNX + PyTorch), SQLite auto-archival
│   ├── application/
│   │   └── analyze_bunch.py    # Use Cases: AnalyzeBunchUseCase, AnalyzeBatchUseCase, SearchSimilarUseCase
│   ├── domain/
│   │   └── models.py           # PalmOilBunch dataclass (core business entity)
│   └── infrastructure/
│       ├── repository.py       # MongoPalmOilRepository (Atlas Vector Search, CRUD)
│       └── vision_service.py   # VertexVisionService (1408-D embeddings, Base64 encoding)
├── demo_app.py                 # Streamlit 5-tab dashboard
├── main.py                     # Root-level uvicorn launcher (DDD wrapper)
├── train_palm.py               # YOLO training script
├── export_raw_tflite.py        # ONNX/TFLite export utility
├── best.onnx                   # YOLO26 ONNX weights (primary engine)
├── best.pt                     # YOLO26 PyTorch weights
├── sawit_tbs.pt                # YOLOv8-Sawit benchmark weights
├── palm_history.db             # Local SQLite audit log
├── batch_outputs/              # Batch job data bundles (manifest + raw images)
├── history_archive/            # Archived images for History Vault
├── feedback/                   # Misclassification feedback data (Human-in-the-Loop)
├── datasets/                   # Labeled training images (Train/Valid/Test)
├── runs/                       # YOLO training logs and output weights
├── requirements.txt            # Python dependencies
├── .env                        # Configuration (secrets, GCP, MongoDB)
└── README.md                   # You are here

Detection Classes (MPOB Standard)

Class	Description	Health Alert
Ripe	Prime harvest condition — maximum OER	❌
Underripe	Harvested before peak — reduces OER	❌
Unripe	Harvested too early — significant yield loss	❌
Overripe	Past peak — potential quality degradation	❌
Abnormal	Disease or structural defect detected	✅ CRITICAL
Empty_Bunch	No fruit present — waste indicator	✅ Warning

---

🔑 Key Design Decisions

• Dual-Engine Inference: ONNX runtime is the primary engine for its ~39ms NMS-free speed. PyTorch (.pt) is retained for high-resolution auditing where standard NMS post-processing is preferred.
• Coordinate Normalization: The batch pipeline stores norm_box (0.0–1.0 ratios) alongside absolute pixel box coordinates. This makes the data contract resolution-agnostic for downstream ERP or vectorization subscribers.
• Graceful Degradation: MongoDB Atlas and Vertex AI connections are established at startup. If they fail (e.g., no billing, no network), the system logs a warning and continues operating in local-only mode. Only cloud-dependent endpoints return errors.
• Human-in-the-Loop: The "Flag Misclassification" feature in the Single Analysis tab saves flagged images and their detection metadata to a local feedback/ folder for future model retraining data collection.
• SQLite Auto-Archival: Every call to /analyze is automatically logged to palm_history.db with the image, detections, engine used, inference/processing latency, and the raw mathematical tensor — enabling a full offline audit trail.

---

📜 License

This project is licensed under the MIT License — see the LICENSE file for details.