palmoil-mobile/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project

palm_oil_mobile is a Flutter app that runs on-device YOLO26 (NMS-free) inference to detect palm oil fresh fruit bunch (FFB) ripeness against MPOB standards, entirely offline (no server/network calls — contrast with frontend/ and server-desktop/ in the parent monorepo, which do remote inference over sockets). Detection classes: Ripe, Unripe, Underripe, Overripe, Abnormal, Empty_Bunch.

Commands

flutter pub get                 # install dependencies
flutter run                     # run on connected device/emulator
flutter build apk                # release Android build
flutter test                     # run all tests
flutter test test/widget_test.dart   # run a single test file
flutter analyze                  # static analysis (flutter_lints, see analysis_options.yaml)

There is no backend to start — the model (assets/best.tflite) and labels (assets/labels.txt) are bundled assets loaded at runtime.

Architecture

Isolate-based inference pipeline

All TFLite inference lives in lib/services/tflite_service.dart. TfliteService spawns a single persistent isolate on initModel() (not one isolate per inference) that owns the Interpreter. The main isolate communicates via SendPort/ReceivePort message passing with commands init, inference_static, inference_stream. This keeps camera-stream frame processing off the UI thread. isIsolateBusy is used as a gate so callers never queue up a second in-flight inference against the same isolate.

Two inference paths decode differently before running the same model:

• Static (_inferenceStaticTask): decodes a file (from camera capture or gallery pick) via the image package, center-crops to square, resizes to 640×640.
• Streaming (_inferenceStreamTask): consumes raw CameraImage planes and manually converts YUV420 (Android) or BGRA8888 (iOS) to RGB before the same crop/resize — see _convertYUV420ToImage for the manual colorspace math.

Detection decoding (_decodeDetections) assumes the model is NMS-free (YOLO26 end-to-end) — output boxes are used directly without a secondary non-max-suppression pass in Dart. Confidence threshold is 0.25 (_confidenceThreshold); box coordinates come back normalized and are re-mapped from crop-space to full-frame-space when crop offsets are provided.

"Point-and-scan" live capture state machine

lib/screens/live_analysis_screen.dart implements an auto-capture flow driven by a DetectionState enum (searching → locking → capturing → cooldown) rather than a manual shutter button:

• A 15-frame sliding window (_detectionHistory) tracks recent high-confidence hits; reaching _requiredHits (5) transitions searching → locking.
• While locking, a 100ms-tick momentum timer (_startLockTimer) accumulates/drains "momentum ticks" — sustained detection advances toward auto-capture, a lost target drains it back to searching.
• On lock, _triggerCapture stops the camera stream, takes a high-res still, re-runs inference on that still (the stream inference is only used for lock detection, not the persisted result), saves to history, then a 3s cooldown timer restarts the stream.

static_capture_screen.dart (manual shutter) and analysis_screen.dart (gallery picker) follow simpler capture → infer → persist → save flows without the state machine.

Persistence

lib/services/database_helper.dart is a singleton wrapping a sqflite table history. Each row is a PalmRecord (lib/models/palm_record.dart): the best (highest-confidence) detection's box/class/confidence are stored as first-class columns, while the full detection list for that frame is serialized as a JSON blob in the detections column (bump _onUpgrade's version check when changing schema). Captured images are copied out of the ephemeral camera/picker temp path into getApplicationDocumentsDirectory() before the DB row is written, so history entries survive app restarts.

Shared UI

lib/widgets/palm_bounding_box.dart (PalmBoundingBox) is the canonical bounding-box overlay with the MPOB color palette (green=Ripe, orange/yellow=Underripe, blue=Unripe, red=Abnormal, gray=Empty_Bunch) and is reused across the live, gallery, and history-detail screens. static_capture_screen.dart still has its own local _buildBoundingBox — prefer PalmBoundingBox for any new box-drawing code.

Screen navigation

main.dart → HomeScreen is a plain card-based Navigator.push menu into four independent screens (AnalysisScreen, StaticCaptureScreen, LiveAnalysisScreen, HistoryScreen) — no named routes, no shared state/provider layer. Each screen owns its own TfliteService and DatabaseHelper instance and must call _tfliteService.dispose() in dispose() to kill its isolate.