# AI-Assisted Smart Data Entry A full-stack prototype for AI-assisted medical claim submission. The system uses OpenAI's GPT-4o-mini Vision model to automatically extract data from uploaded medical receipt images and populate a structured claim form, reducing manual data entry effort for HR and finance teams. --- ## Features - **Dual AI Extraction Tracks** - **V1 – Smart Auto-Fill**: Sends the receipt image to a fixed structured schema (Pydantic-enforced). Returns fields like provider name, date, amount, items, confidence score, and flags receipts needing manual review. - **V2 – Template-Fill**: Accepts a configurable field template and fills only the fields explicitly present in the image. Stricter — returns `null` for any field that cannot be confirmed. - **Visual AI Indicators**: Form fields are colour-coded to distinguish V1-filled (blue) vs V2-filled (purple) vs manually entered values. - **SQLite Persistence**: Claims and users are stored in a local `data.db` SQLite database via SQLAlchemy ORM. - **Allowance Tracking**: Each submitted claim deducts from the user's medical allowance. Deleting a claim refunds the deducted amount. - **Confidence Scoring & Manual Review Flags**: V1 extraction returns a confidence score and a `needs_manual_review` flag for ambiguous receipts. - **Image Compression**: Uploaded images are resized and compressed (max 2000×2000, JPEG quality 85) before being sent to the API. - **Angular Frontend**: Single-page Angular app with a claim submission form and a claims dashboard. --- ## Project Structure ``` AI-Assisted Smart Data Entry/ ├── backend/ │ ├── __init__.py │ ├── main.py # FastAPI app, routes, startup seeding │ ├── database.py # SQLAlchemy engine & session setup (SQLite) │ ├── models.py # SQLAlchemy ORM models (User, Claim) │ ├── schemas.py # Pydantic schemas for request/response validation │ ├── crud.py # Database CRUD operations and business logic │ └── services/ │ └── openai_service.py # OpenAI Vision API calls (V1 & V2) ├── ai-data-entry-ui/ # Angular 17+ frontend │ └── src/ │ ├── index.html │ ├── main.ts │ ├── styles.css │ └── app/ │ ├── app.ts │ ├── app.routes.ts │ ├── app.config.ts │ ├── components/ │ │ ├── claim-form/ # Claim submission form with AI fill buttons │ │ └── claims-dashboard/ # Dashboard listing all submitted claims │ └── services/ │ ├── extraction.ts # TypeScript interfaces (models) │ ├── extraction.service.ts # HTTP service for all API calls │ └── session.service.ts # In-memory session / current user state ├── sample_medical_receipts/ # Sample images for testing ├── data.db # SQLite database (auto-created on first run) ├── requirements.txt ├── .env # API key (not tracked by git) └── .gitignore ``` --- ## Setup ### Prerequisites - Python 3.10+ - Node.js 18+ and npm - OpenAI API key ### Backend (FastAPI) 1. **Create and activate a virtual environment**: ```bash python -m venv venv # Windows .\venv\Scripts\activate # Mac/Linux source venv/bin/activate ``` 2. **Install dependencies**: ```bash pip install -r requirements.txt ``` 3. **Configure environment variables**: Create a `.env` file in the project root: ```env OPENAI_API_KEY=your_openai_api_key_here ``` 4. **Run the server**: ```bash uvicorn backend.main:app --reload ``` The API will be available at `http://localhost:8000`. On first startup, the database is created and a **Demo User** (ID: `demo-user-123`, allowance: MYR 5,000) is automatically seeded. ### Frontend (Angular) 1. **Install dependencies**: ```bash cd ai-data-entry-ui npm install --legacy-peer-deps ``` 2. **Run the development server**: ```bash ng serve ``` The UI will be available at `http://localhost:4200`. --- ## API Endpoints ### `GET /health` Returns server health status. --- ### `GET /api/v1/users` Returns a list of all users. **Response**: `UserAccount[]` --- ### `POST /api/v1/users` Creates a new user. **Request Body** (JSON): ```json { "id": "string", "name": "string", "department": "string", "medical_allowance": 5000.0 } ``` --- ### `POST /api/v1/extract` — V1 AI Extraction Extracts structured data from a medical receipt image using a fixed schema. **Request**: `multipart/form-data` | Field | Type | Description | |---|---|---| | `file` | Image file | The medical receipt image | | `user_name` | string (optional) | Employee name (default: `Demo User`) | | `department` | string (optional) | Employee department (default: `Operations`) | **Response** (`ExtractionResponse`): ```json { "provider_name": "Klinik Sejahtera", "visit_date": "2024-03-15", "total_amount": 85.00, "currency": "MYR", "items": ["Consultation", "Medicine"], "confidence_score": 0.92, "needs_manual_review": false, "ai_reasoning": "Clinic name and stamp clearly visible on letterhead.", "receipt_ref_no": "INV-00123", "clinic_reg_no": "MOH/12345", "claim_category": "General", "diagnosis_brief": "Fever consultation and prescribed medication." } ``` --- ### `POST /api/v2/fill-template` — V2 Template-Driven Fill Fills a user-defined field template from the receipt image. Only returns values explicitly found in the image. **Request**: `multipart/form-data` | Field | Type | Description | |---|---|---| | `file` | Image file | The medical receipt image | | `template_fields` | JSON string | Map of `fieldKey → description` | | `user_name` | string (optional) | Employee name | | `department` | string (optional) | Employee department | **Example `template_fields`**: ```json { "provider_name": "Name of clinic/hospital", "visit_date": "Date of treatment (YYYY-MM-DD)", "amount_spent": "Total amount paid", "receipt_ref_no": "Invoice or reference number" } ``` **Response** (`V2TemplateResponse`): ```json { "filled_data": [ { "key": "provider_name", "value": "Klinik Sejahtera" }, { "key": "amount_spent", "value": 85.00 } ], "unfilled_fields": ["receipt_ref_no"] } ``` --- ### `POST /api/v1/claims` Submits a medical claim. Deducts the claimable amount from the user's allowance (capped at remaining balance). **Request Header**: `user-id: ` **Request Body** (`ClaimSubmission`): ```json { "provider_name": "string", "visit_date": "YYYY-MM-DD", "amount_spent": 85.00, "currency": "MYR", "treatment_type": "Outpatient", "cost_center": "CC-001", "declaration_signed": true, "extraction_data": { } } ``` --- ### `GET /api/v1/claims` Returns all submitted claims (all users). --- ### `DELETE /api/v1/claims/{claim_id}` Deletes a claim and refunds the deducted amount back to the user's allowance. --- ## Data Models ### Backend (SQLAlchemy) | Model | Fields | |---|---| | `User` | `id`, `name`, `department`, `medical_allowance` | | `Claim` | `id`, `timestamp`, `user_id` (FK), `amount_spent`, `amount_claimed`, `provider_name`, `visit_date`, `treatment_type`, `cost_center`, `declaration_signed`, `extraction_data` (JSON text) | ### Frontend (TypeScript Interfaces) Defined in `extraction.ts`: `UserAccount`, `ExtractionResponse`, `ClaimSubmission`, `ClaimRecord`, `V2Field`, `V2TemplateResponse`. --- ## Frontend Components ### `ClaimFormComponent` The main data entry view. Features: - Drag-and-drop or click-to-upload receipt image with live preview. - **"Auto-Fill with AI" (V1)** button — calls `/api/v1/extract` and populates the form. - **"Template Fill (V2)"** button — calls `/api/v2/fill-template` and populates fields found in the image. - Colour-coded field borders: **blue** = filled by V1, **purple** = filled by V2. - Real-time claimable amount calculation (capped by remaining allowance). - Raw JSON response toggle for debugging. - Declaration checkbox required before submission. ### `ClaimsDashboardComponent` Lists all submitted claims with delete functionality and displays the current user's remaining medical allowance. --- ## Dependencies **Backend** (`requirements.txt`): - `fastapi` - `uvicorn` - `openai>=1.50.0` - `python-dotenv` - `python-multipart` - `pydantic>=2.0` - `Pillow` - `sqlalchemy` **Frontend**: Angular 17+, `lucide-angular` (icons), Angular Reactive Forms, Angular Router.