AI-Assisted-Smart-Data-Entry-PoC/README.md

AI-Assisted Smart Data Entry

A full-stack prototype for AI-assisted medical claim submission. The system uses OpenAI's GPT-4o-mini Vision model to automatically extract data from uploaded medical receipt images and populate a structured claim form, reducing manual data entry effort for HR and finance teams.

---

Features

• Dual AI Extraction Tracks
  • V1 – Smart Auto-Fill: Sends the receipt image to a fixed structured schema (Pydantic-enforced). Returns fields like provider name, date, amount, items, confidence score, and flags receipts needing manual review.
  • V2 – Template-Fill: Accepts a configurable field template and fills only the fields explicitly present in the image. Stricter — returns null for any field that cannot be confirmed.
• Visual AI Indicators: Form fields are colour-coded to distinguish V1-filled (blue) vs V2-filled (purple) vs manually entered values.
• SQLite Persistence: Claims and users are stored in a local data.db SQLite database via SQLAlchemy ORM.
• Allowance Tracking: Each submitted claim deducts from the user's medical allowance. Deleting a claim refunds the deducted amount.
• Confidence Scoring & Manual Review Flags: V1 extraction returns a confidence score and a needs_manual_review flag for ambiguous receipts.
• Image Compression: Uploaded images are resized and compressed (max 2000×2000, JPEG quality 85) before being sent to the API.
• Angular Frontend: Single-page Angular app with a claim submission form and a claims dashboard.

---

Project Structure

AI-Assisted Smart Data Entry/
├── backend/
│   ├── __init__.py
│   ├── main.py            # FastAPI app, routes, startup seeding
│   ├── database.py        # SQLAlchemy engine & session setup (SQLite)
│   ├── models.py          # SQLAlchemy ORM models (User, Claim)
│   ├── schemas.py         # Pydantic schemas for request/response validation
│   ├── crud.py            # Database CRUD operations and business logic
│   └── services/
│       └── openai_service.py  # OpenAI Vision API calls (V1 & V2)
├── ai-data-entry-ui/          # Angular 17+ frontend
│   └── src/
│       ├── index.html
│       ├── main.ts
│       ├── styles.css
│       └── app/
│           ├── app.ts
│           ├── app.routes.ts
│           ├── app.config.ts
│           ├── components/
│           │   ├── claim-form/         # Claim submission form with AI fill buttons
│           │   └── claims-dashboard/   # Dashboard listing all submitted claims
│           └── services/
│               ├── extraction.ts           # TypeScript interfaces (models)
│               ├── extraction.service.ts   # HTTP service for all API calls
│               └── session.service.ts      # In-memory session / current user state
├── sample_medical_receipts/   # Sample images for testing
├── data.db                    # SQLite database (auto-created on first run)
├── requirements.txt
├── .env                       # API key (not tracked by git)
└── .gitignore

---

Setup

Prerequisites

• Python 3.10+
• Node.js 18+ and npm
• OpenAI API key

Backend (FastAPI)

1. Create and activate a virtual environment: ```bash python -m venv venv

# Windows .\venv\Scripts\activate

# Mac/Linux source venv/bin/activate

2. **Install dependencies**:
   ```bash
   pip install -r requirements.txt

1. Configure environment variables: Create a .env file in the project root:

   OPENAI_API_KEY=your_openai_api_key_here
   

   1. Run the server: bash uvicorn backend.main:app --reload

The API will be available at http://localhost:8000.
On first startup, the database is created and a Demo User (ID: demo-user-123, allowance: MYR 5,000) is automatically seeded.

Frontend (Angular)

1. Install dependencies:

   cd ai-data-entry-ui
   npm install --legacy-peer-deps
   

   1. Run the development server: bash ng serve

The UI will be available at http://localhost:4200.

---

API Endpoints

GET /health

Returns server health status.

---

GET /api/v1/users

Returns a list of all users.

Response: UserAccount[]

---

POST /api/v1/users

Creates a new user.

Request Body (JSON):

{
  "id": "string",
  "name": "string",
  "department": "string",
  "medical_allowance": 5000.0
}

---

POST /api/v1/extract — V1 AI Extraction

Extracts structured data from a medical receipt image using a fixed schema.

Request: multipart/form-data | Field | Type | Description | |---|---|---| | file | Image file | The medical receipt image | | user_name | string (optional) | Employee name (default: Demo User) | | department | string (optional) | Employee department (default: Operations) |

Response (ExtractionResponse):

{
  "provider_name": "Klinik Sejahtera",
  "visit_date": "2024-03-15",
  "total_amount": 85.00,
  "currency": "MYR",
  "items": ["Consultation", "Medicine"],
  "confidence_score": 0.92,
  "needs_manual_review": false,
  "ai_reasoning": "Clinic name and stamp clearly visible on letterhead.",
  "receipt_ref_no": "INV-00123",
  "clinic_reg_no": "MOH/12345",
  "claim_category": "General",
  "diagnosis_brief": "Fever consultation and prescribed medication."
}

---

POST /api/v2/fill-template — V2 Template-Driven Fill

Fills a user-defined field template from the receipt image. Only returns values explicitly found in the image.

Request: multipart/form-data | Field | Type | Description | |---|---|---| | file | Image file | The medical receipt image | | template_fields | JSON string | Map of fieldKey → description | | user_name | string (optional) | Employee name | | department | string (optional) | Employee department |

Example template_fields:

{
  "provider_name": "Name of clinic/hospital",
  "visit_date": "Date of treatment (YYYY-MM-DD)",
  "amount_spent": "Total amount paid",
  "receipt_ref_no": "Invoice or reference number"
}

Response (V2TemplateResponse):

{
  "filled_data": [
    { "key": "provider_name", "value": "Klinik Sejahtera" },
    { "key": "amount_spent", "value": 85.00 }
  ],
  "unfilled_fields": ["receipt_ref_no"]
}

---

POST /api/v1/claims

Submits a medical claim. Deducts the claimable amount from the user's allowance (capped at remaining balance).

Request Header: user-id: <user_id>

Request Body (ClaimSubmission):

{
  "provider_name": "string",
  "visit_date": "YYYY-MM-DD",
  "amount_spent": 85.00,
  "currency": "MYR",
  "treatment_type": "Outpatient",
  "cost_center": "CC-001",
  "declaration_signed": true,
  "extraction_data": { }
}

---

GET /api/v1/claims

Returns all submitted claims (all users).

---

DELETE /api/v1/claims/{claim_id}

Deletes a claim and refunds the deducted amount back to the user's allowance.

---

Data Models

Backend (SQLAlchemy)

Model	Fields
User	id, name, department, medical_allowance
Claim	id, timestamp, user_id (FK), amount_spent, amount_claimed, provider_name, visit_date, treatment_type, cost_center, declaration_signed, extraction_data (JSON text)

Frontend (TypeScript Interfaces)

Defined in extraction.ts: UserAccount, ExtractionResponse, ClaimSubmission, ClaimRecord, V2Field, V2TemplateResponse.

---

Frontend Components

ClaimFormComponent

The main data entry view. Features:

• Drag-and-drop or click-to-upload receipt image with live preview.
• "Auto-Fill with AI" (V1) button — calls /api/v1/extract and populates the form.
• "Template Fill (V2)" button — calls /api/v2/fill-template and populates fields found in the image.
• Colour-coded field borders: blue = filled by V1, purple = filled by V2.
• Real-time claimable amount calculation (capped by remaining allowance).
• Raw JSON response toggle for debugging.
• Declaration checkbox required before submission.

ClaimsDashboardComponent

Lists all submitted claims with delete functionality and displays the current user's remaining medical allowance.

---

Dependencies

Backend (requirements.txt):

• fastapi
• uvicorn
• openai>=1.50.0
• python-dotenv
• python-multipart
• pydantic>=2.0
• Pillow
• sqlalchemy

Frontend: Angular 17+, lucide-angular (icons), Angular Reactive Forms, Angular Router.