|
|
@@ -1,82 +1,282 @@
|
|
|
-# AI-Assisted Smart Data Entry Backend
|
|
|
+# AI-Assisted Smart Data Entry
|
|
|
|
|
|
-This project provides a FastAPI-based backend for extracting structured data from medical receipts using OpenAI's GPT-4o model with structured outputs.
|
|
|
+A full-stack prototype for AI-assisted medical claim submission. The system uses OpenAI's GPT-4o-mini Vision model to automatically extract data from uploaded medical receipt images and populate a structured claim form, reducing manual data entry effort for HR and finance teams.
|
|
|
+
|
|
|
+---
|
|
|
|
|
|
## Features
|
|
|
|
|
|
-- **FastAPI**: Modern, fast web framework for building APIs.
|
|
|
-- **Structured Extraction**: Uses OpenAI's `beta.chat.completions.parse` to ensure extracted data adheres to a Pydantic schema.
|
|
|
-- **CORS Enabled**: Configured to allow access from frontend applications (e.g., Angular).
|
|
|
-- **Health Check**: Simple endpoint to verify server status.
|
|
|
+- **Dual AI Extraction Tracks**
|
|
|
+ - **V1 – Smart Auto-Fill**: Sends the receipt image to a fixed structured schema (Pydantic-enforced). Returns fields like provider name, date, amount, items, confidence score, and flags receipts needing manual review.
|
|
|
+ - **V2 – Template-Fill**: Accepts a configurable field template and fills only the fields explicitly present in the image. Stricter — returns `null` for any field that cannot be confirmed.
|
|
|
+- **Visual AI Indicators**: Form fields are colour-coded to distinguish V1-filled (blue) vs V2-filled (purple) vs manually entered values.
|
|
|
+- **SQLite Persistence**: Claims and users are stored in a local `data.db` SQLite database via SQLAlchemy ORM.
|
|
|
+- **Allowance Tracking**: Each submitted claim deducts from the user's medical allowance. Deleting a claim refunds the deducted amount.
|
|
|
+- **Confidence Scoring & Manual Review Flags**: V1 extraction returns a confidence score and a `needs_manual_review` flag for ambiguous receipts.
|
|
|
+- **Image Compression**: Uploaded images are resized and compressed (max 2000×2000, JPEG quality 85) before being sent to the API.
|
|
|
+- **Angular Frontend**: Single-page Angular app with a claim submission form and a claims dashboard.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Project Structure
|
|
|
+
|
|
|
+```
|
|
|
+AI-Assisted Smart Data Entry/
|
|
|
+├── backend/
|
|
|
+│ ├── __init__.py
|
|
|
+│ ├── main.py # FastAPI app, routes, startup seeding
|
|
|
+│ ├── database.py # SQLAlchemy engine & session setup (SQLite)
|
|
|
+│ ├── models.py # SQLAlchemy ORM models (User, Claim)
|
|
|
+│ ├── schemas.py # Pydantic schemas for request/response validation
|
|
|
+│ ├── crud.py # Database CRUD operations and business logic
|
|
|
+│ └── services/
|
|
|
+│ └── openai_service.py # OpenAI Vision API calls (V1 & V2)
|
|
|
+├── ai-data-entry-ui/ # Angular 17+ frontend
|
|
|
+│ └── src/
|
|
|
+│ ├── index.html
|
|
|
+│ ├── main.ts
|
|
|
+│ ├── styles.css
|
|
|
+│ └── app/
|
|
|
+│ ├── app.ts
|
|
|
+│ ├── app.routes.ts
|
|
|
+│ ├── app.config.ts
|
|
|
+│ ├── components/
|
|
|
+│ │ ├── claim-form/ # Claim submission form with AI fill buttons
|
|
|
+│ │ └── claims-dashboard/ # Dashboard listing all submitted claims
|
|
|
+│ └── services/
|
|
|
+│ ├── extraction.ts # TypeScript interfaces (models)
|
|
|
+│ ├── extraction.service.ts # HTTP service for all API calls
|
|
|
+│ └── session.service.ts # In-memory session / current user state
|
|
|
+├── sample_medical_receipts/ # Sample images for testing
|
|
|
+├── data.db # SQLite database (auto-created on first run)
|
|
|
+├── requirements.txt
|
|
|
+├── .env # API key (not tracked by git)
|
|
|
+└── .gitignore
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
|
|
|
## Setup
|
|
|
|
|
|
+### Prerequisites
|
|
|
+
|
|
|
+- Python 3.10+
|
|
|
+- Node.js 18+ and npm
|
|
|
+- OpenAI API key
|
|
|
+
|
|
|
### Backend (FastAPI)
|
|
|
|
|
|
-1. **Initialize Virtual Environment**:
|
|
|
+1. **Create and activate a virtual environment**:
|
|
|
```bash
|
|
|
- # Create environment
|
|
|
python -m venv venv
|
|
|
|
|
|
- # Activate environment (Windows)
|
|
|
+ # Windows
|
|
|
.\venv\Scripts\activate
|
|
|
|
|
|
- # Activate environment (Mac/Linux)
|
|
|
- # source venv/bin/activate
|
|
|
+ # Mac/Linux
|
|
|
+ source venv/bin/activate
|
|
|
```
|
|
|
|
|
|
-2. **Install Dependencies**:
|
|
|
+2. **Install dependencies**:
|
|
|
```bash
|
|
|
pip install -r requirements.txt
|
|
|
```
|
|
|
|
|
|
-2. **Configure Environment**:
|
|
|
- Create a `.env` file in the root directory and add your OpenAI API key:
|
|
|
+3. **Configure environment variables**:
|
|
|
+ Create a `.env` file in the project root:
|
|
|
```env
|
|
|
- OPENAI_API_KEY=your_api_key_here
|
|
|
+ OPENAI_API_KEY=your_openai_api_key_here
|
|
|
```
|
|
|
|
|
|
-3. **Run the Server**:
|
|
|
+4. **Run the server**:
|
|
|
```bash
|
|
|
uvicorn backend.main:app --reload
|
|
|
```
|
|
|
+ The API will be available at `http://localhost:8000`.
|
|
|
+ On first startup, the database is created and a **Demo User** (ID: `demo-user-123`, allowance: MYR 5,000) is automatically seeded.
|
|
|
|
|
|
### Frontend (Angular)
|
|
|
|
|
|
-1. **Install Dependencies**:
|
|
|
+1. **Install dependencies**:
|
|
|
```bash
|
|
|
cd ai-data-entry-ui
|
|
|
npm install --legacy-peer-deps
|
|
|
```
|
|
|
|
|
|
-2. **Run the UI**:
|
|
|
+2. **Run the development server**:
|
|
|
```bash
|
|
|
- npm start
|
|
|
+ ng serve
|
|
|
```
|
|
|
+ The UI will be available at `http://localhost:4200`.
|
|
|
+
|
|
|
+---
|
|
|
|
|
|
## API Endpoints
|
|
|
|
|
|
### `GET /health`
|
|
|
-Returns the status of the server.
|
|
|
+Returns server health status.
|
|
|
|
|
|
-### `POST /api/v1/extract`
|
|
|
-Extracts data from a medical receipt image.
|
|
|
+---
|
|
|
|
|
|
-**Request Body**:
|
|
|
-- `file`: Multipart image file.
|
|
|
-- `user_name` (optional): Name of the employee.
|
|
|
-- `department` (optional): Department of the employee.
|
|
|
+### `GET /api/v1/users`
|
|
|
+Returns a list of all users.
|
|
|
|
|
|
-**Response**:
|
|
|
-Returns a structured JSON matching the `ExtractionResponse` schema.
|
|
|
+**Response**: `UserAccount[]`
|
|
|
|
|
|
-## Project Structure
|
|
|
+---
|
|
|
+
|
|
|
+### `POST /api/v1/users`
|
|
|
+Creates a new user.
|
|
|
+
|
|
|
+**Request Body** (JSON):
|
|
|
+```json
|
|
|
+{
|
|
|
+ "id": "string",
|
|
|
+ "name": "string",
|
|
|
+ "department": "string",
|
|
|
+ "medical_allowance": 5000.0
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+### `POST /api/v1/extract` — V1 AI Extraction
|
|
|
+Extracts structured data from a medical receipt image using a fixed schema.
|
|
|
+
|
|
|
+**Request**: `multipart/form-data`
|
|
|
+| Field | Type | Description |
|
|
|
+|---|---|---|
|
|
|
+| `file` | Image file | The medical receipt image |
|
|
|
+| `user_name` | string (optional) | Employee name (default: `Demo User`) |
|
|
|
+| `department` | string (optional) | Employee department (default: `Operations`) |
|
|
|
+
|
|
|
+**Response** (`ExtractionResponse`):
|
|
|
+```json
|
|
|
+{
|
|
|
+ "provider_name": "Klinik Sejahtera",
|
|
|
+ "visit_date": "2024-03-15",
|
|
|
+ "total_amount": 85.00,
|
|
|
+ "currency": "MYR",
|
|
|
+ "items": ["Consultation", "Medicine"],
|
|
|
+ "confidence_score": 0.92,
|
|
|
+ "needs_manual_review": false,
|
|
|
+ "ai_reasoning": "Clinic name and stamp clearly visible on letterhead.",
|
|
|
+ "receipt_ref_no": "INV-00123",
|
|
|
+ "clinic_reg_no": "MOH/12345",
|
|
|
+ "claim_category": "General",
|
|
|
+ "diagnosis_brief": "Fever consultation and prescribed medication."
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+### `POST /api/v2/fill-template` — V2 Template-Driven Fill
|
|
|
+Fills a user-defined field template from the receipt image. Only returns values explicitly found in the image.
|
|
|
+
|
|
|
+**Request**: `multipart/form-data`
|
|
|
+| Field | Type | Description |
|
|
|
+|---|---|---|
|
|
|
+| `file` | Image file | The medical receipt image |
|
|
|
+| `template_fields` | JSON string | Map of `fieldKey → description` |
|
|
|
+| `user_name` | string (optional) | Employee name |
|
|
|
+| `department` | string (optional) | Employee department |
|
|
|
+
|
|
|
+**Example `template_fields`**:
|
|
|
+```json
|
|
|
+{
|
|
|
+ "provider_name": "Name of clinic/hospital",
|
|
|
+ "visit_date": "Date of treatment (YYYY-MM-DD)",
|
|
|
+ "amount_spent": "Total amount paid",
|
|
|
+ "receipt_ref_no": "Invoice or reference number"
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+**Response** (`V2TemplateResponse`):
|
|
|
+```json
|
|
|
+{
|
|
|
+ "filled_data": [
|
|
|
+ { "key": "provider_name", "value": "Klinik Sejahtera" },
|
|
|
+ { "key": "amount_spent", "value": 85.00 }
|
|
|
+ ],
|
|
|
+ "unfilled_fields": ["receipt_ref_no"]
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+### `POST /api/v1/claims`
|
|
|
+Submits a medical claim. Deducts the claimable amount from the user's allowance (capped at remaining balance).
|
|
|
+
|
|
|
+**Request Header**: `user-id: <user_id>`
|
|
|
+
|
|
|
+**Request Body** (`ClaimSubmission`):
|
|
|
+```json
|
|
|
+{
|
|
|
+ "provider_name": "string",
|
|
|
+ "visit_date": "YYYY-MM-DD",
|
|
|
+ "amount_spent": 85.00,
|
|
|
+ "currency": "MYR",
|
|
|
+ "treatment_type": "Outpatient",
|
|
|
+ "cost_center": "CC-001",
|
|
|
+ "declaration_signed": true,
|
|
|
+ "extraction_data": { }
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+### `GET /api/v1/claims`
|
|
|
+Returns all submitted claims (all users).
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+### `DELETE /api/v1/claims/{claim_id}`
|
|
|
+Deletes a claim and refunds the deducted amount back to the user's allowance.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Data Models
|
|
|
+
|
|
|
+### Backend (SQLAlchemy)
|
|
|
+
|
|
|
+| Model | Fields |
|
|
|
+|---|---|
|
|
|
+| `User` | `id`, `name`, `department`, `medical_allowance` |
|
|
|
+| `Claim` | `id`, `timestamp`, `user_id` (FK), `amount_spent`, `amount_claimed`, `provider_name`, `visit_date`, `treatment_type`, `cost_center`, `declaration_signed`, `extraction_data` (JSON text) |
|
|
|
+
|
|
|
+### Frontend (TypeScript Interfaces)
|
|
|
+
|
|
|
+Defined in `extraction.ts`: `UserAccount`, `ExtractionResponse`, `ClaimSubmission`, `ClaimRecord`, `V2Field`, `V2TemplateResponse`.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Frontend Components
|
|
|
+
|
|
|
+### `ClaimFormComponent`
|
|
|
+The main data entry view. Features:
|
|
|
+- Drag-and-drop or click-to-upload receipt image with live preview.
|
|
|
+- **"Auto-Fill with AI" (V1)** button — calls `/api/v1/extract` and populates the form.
|
|
|
+- **"Template Fill (V2)"** button — calls `/api/v2/fill-template` and populates fields found in the image.
|
|
|
+- Colour-coded field borders: **blue** = filled by V1, **purple** = filled by V2.
|
|
|
+- Real-time claimable amount calculation (capped by remaining allowance).
|
|
|
+- Raw JSON response toggle for debugging.
|
|
|
+- Declaration checkbox required before submission.
|
|
|
+
|
|
|
+### `ClaimsDashboardComponent`
|
|
|
+Lists all submitted claims with delete functionality and displays the current user's remaining medical allowance.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Dependencies
|
|
|
+
|
|
|
+**Backend** (`requirements.txt`):
|
|
|
+- `fastapi`
|
|
|
+- `uvicorn`
|
|
|
+- `openai>=1.50.0`
|
|
|
+- `python-dotenv`
|
|
|
+- `python-multipart`
|
|
|
+- `pydantic>=2.0`
|
|
|
+- `Pillow`
|
|
|
+- `sqlalchemy`
|
|
|
|
|
|
-- `backend/`
|
|
|
- - `main.py`: Entry point and API endpoints.
|
|
|
- - `schemas.py`: Pydantic models for data validation and structured output.
|
|
|
- - `services/`:
|
|
|
- - `openai_service.py`: Logic for interacting with OpenAI API.
|
|
|
-- `ai-data-entry-ui/`: Angular frontend.
|
|
|
-- `requirements.txt`: Python dependencies.
|
|
|
-- `.env`: Environment variables (not tracked by git).
|
|
|
+**Frontend**: Angular 17+, `lucide-angular` (icons), Angular Reactive Forms, Angular Router.
|
Dr-Swopt