## Summary
Add Fetch Request pipeline UI with real-time SSE progress tracking, ambiguity resolution, enhanced list page with filtering/retry, and detail page with stepper/event feed. Includes new API primitives (`api.patch`, `usePatch` hook) and extensive type definitions for SSE events, ambiguity resolution, and pipeline statuses.
## Changes by file
### `react-openapi/api/client.ts`
- Added `api.patch()` method (delegates to `AxiosInstance.patch`)
### `react-openapi/hooks/useResource.ts`
- Added `usePatch()` mutation hook — sends PATCH `/{endpoint}/{id}` with partial data, invalidates list + detail query caches on success
### `src/FetchRequestDetail.tsx` **(new, +675 lines)**
Full detail page for a single fetch request, composed of:
- **Header** — status chip, account name, source type (File/Email), date range, created/completed timestamps
- **Progress bar** — `LinearProgress` with percentage computed via `computeProgressPercent()`:
- Extract phase: snaps to 10% on first `raw_lines`/`txn_blocks` event
- Raw Expense: ratio of `displayParsedCount` / `txnBlockCount` × 20%
- Enrich: ratio of `stepStats.enrich_count` / `txnDictCount` × 50%
- Save: ratio of `stepStats.save_count` / `txnDictCount` × 20%
- **Retry section** — retry count bar (`retryCount/RETRY_MAX`) + "Retry" button if failed & not exhausted
- **Error/success alerts** — `error_message` display, max-retries info
- **Pipeline Stepper** — 4-step (`Extract` → `Raw Expense` → `Enrich` → `Save`) with custom icons per state: completed (CheckCircle, green), active (CircularProgress, animating), paused (WarningAmber, amber), failed (ErrorIcon, red), inactive (step number, grey). Step count labels shown below each label (e.g. `150/246`, `100/246`)
- **Progress Events feed** — auto-scrolling list of SSE events with deduplication:
- Only latest `progress` event per step (`txn_dicts`, `enrich`, `save_expenses`)
- `started` events hidden when a terminal event (`completed`/`skipped`/`paused`/`failed`) follows
- `load_content` events excluded entirely
- Connection status indicator (green dot / red dot)
- **Ambiguity Resolution** — When pipeline is `paused`, shows ambiguity cards:
- Raw line in monospace code block
- OCR amount/balance (struck through), previous balance
- Candidate buttons with credit/debit coloring (green for positive, red for negative)
- Resolved state shows green alert with chosen values
- "All resolved" vs "Pipeline paused" alerts
- **SSE connection** — `EventSource` to `{baseUrl}/fetch-requests/{id}/events`:
- Tracks `progress`, `completed`, `paused`, `failed` events
- On `paused`: refetch request + ambiguities
- On `failed`: refetch request + show error snackbar from `message.error`
- On `completed`/`resume_extract`: refetch request
- Cleans up on unmount
- **Snackbar** — pipeline failure notification (6s, bottom-center)
### `src/FetchRequests.tsx` **(+347/−73 lines)**
Major enhancement to the list page:
- **New filter bar** (replaces plain list header):
- Status multi-select (pending, processing, paused, raw_expenses_done, enriched_done, completed, failed)
- Account name text filter
- Source type toggle (All / File / Email)
- Refresh button
- **Account autocomplete** — fetches accounts list via `useResourceByName("accounts")`, provides dropdown
- **Format dropdown** — driven by `resourceOverrides` config (`fetchRes.fields.source.schema.format.options`), fallback `["axis", "icici_ocr"]`
- **Date pickers** — changed from `datetime-local` to `date`, capped at today's date
- **Navigation** — on create, navigates to `/fetch-requests/{id}` via `useNavigate`
- **Row actions**: retry (failed, retry_count < 3), navigate to detail (paused), delete with confirmation dialog
- **Copy fingerprint** — icon button copies to clipboard with snackbar confirmation
- **Sorting** — table sorted by `created_at` descending
- **Table columns** — changed from `[Fingerprint, Source, Account, Status, Created, Actions]` to `[ID, Account, Source, Date Range, Status, Retries, Created, Actions]`
- **Retry count display** — shows `retry_count/RETRY_MAX` when >0, otherwise `—`
- **Status tooltip** — shows `error_message` on hover when present
- **Status icons** — new `statusIcons` map: ScheduleIcon (pending), CircularProgress (processing), WarningAmber (paused), HourglassEmpty (raw_expenses_done/enriched_done), CheckCircle (completed), ErrorIcon (failed)
- **Error handling** — 409 conflict detection (duplicate fingerprint), 422 validation via `formatApiError()`
- **`handleRetry`** — PATCH `{status: "pending"}` on failed requests, success/error snackbar
### `src/features/fetch-requests/fetch-requests.models.ts` **(+97 lines)**
New types and helpers:
- Added `"paused"` to `FetchRequestStatus`
- `FileSource`: added `raw_lines`, `txn_blocks`, `txn_dicts`, `txn_dict_count`, `txn_dicts_count`
- `EmailSource`: added `txn_dict_count`, `txn_dicts_count`
- `FetchRequest` added `retry_count`
- **New interfaces**: `FetchRequestUpdate`, `AmbiguityCandidate`, `PendingAmbiguity`, `ResolveAmbiguityPayload`, `FetchRequestFilters`
- **SSE types**:
- `SSEEventStep`: `load_content | raw_lines | txn_blocks | txn_dicts | resume_extract | extract | paused | complete | enrich | save_expenses | pipeline`
- `SSEEventStatus`: `started | completed | skipped | paused | progress | failed`
- `ProgressMessage`: `lines? | blocks? | count? | unit? | raw_ocr_line? | error?`
- `SSEEvent: { step, status, message }`
- **Helpers**: `formatApiError()` — parses FastAPI 422 validation detail arrays (`"Missing: field_name"`), `RETRY_MAX = 3`
### `src/features/fetch-requests/index.ts` **(+13 lines)**
Barrel exports for all new types (`FetchRequestUpdate`, `FetchRequestFilters`, `PendingAmbiguity`, `AmbiguityCandidate`, `ResolveAmbiguityPayload`, `SSEEvent`, `SSEEventStep`, `SSEEventStatus`, `ProgressMessage`), value exports (`RETRY_MAX`, `formatApiError`), and new hooks (`useUpdateFetchRequest`, `useFetchRequestAmbiguities`, `useResolveAmbiguity`)
### `src/features/fetch-requests/useFetchRequests.ts` **(+49 lines)**
Added hooks:
- `useUpdateFetchRequest()` → `usePatch("fetch-requests")`
- `useFetchRequestAmbiguities(fetchRequestId)` → `useQuery` for `GET /fetch-requests/{id}/ambiguities`
- `useResolveAmbiguity()` → `useMutation` for `POST /ambiguities/{id}/resolve` with cache invalidation of both ambiguities and detail queries
### `src/main.jsx` **(+2 lines)**
- Added import for `FetchRequestDetail`
- Added route `{ path: "/fetch-requests/:id", component: FetchRequestDetail, headerTitle: "Fetch Request" }`
Reviewed-on: #9
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
134 lines
3.0 KiB
TypeScript
134 lines
3.0 KiB
TypeScript
export type FetchRequestStatus =
|
|
| "pending"
|
|
| "processing"
|
|
| "paused"
|
|
| "raw_expenses_done"
|
|
| "enriched_done"
|
|
| "completed"
|
|
| "failed";
|
|
|
|
export interface FileSource {
|
|
path: string;
|
|
format: string;
|
|
raw_lines?: string[];
|
|
txn_blocks?: Record<string, any>;
|
|
txn_dicts?: Record<string, any>[];
|
|
txn_dict_count?: number;
|
|
txn_dicts_count?: number;
|
|
}
|
|
|
|
export interface EmailSource {
|
|
format: string;
|
|
from_email?: string;
|
|
subject?: string;
|
|
raw_terms?: string[];
|
|
txn_dict_count?: number;
|
|
txn_dicts_count?: number;
|
|
}
|
|
|
|
export interface FetchRequestCreate {
|
|
source: FileSource | EmailSource;
|
|
account_name: string;
|
|
payor_username?: string;
|
|
start_date?: string;
|
|
end_date?: string;
|
|
}
|
|
|
|
export interface FetchRequestUpdate {
|
|
status?: FetchRequestStatus;
|
|
error_message?: string | null;
|
|
}
|
|
|
|
export interface FetchRequest extends FetchRequestCreate {
|
|
id: string;
|
|
status: FetchRequestStatus;
|
|
fingerprint: string;
|
|
completed_at?: string | null;
|
|
error_message?: string | null;
|
|
retry_count?: number;
|
|
created_at: string;
|
|
}
|
|
|
|
export interface UploadResult {
|
|
original_filename: string;
|
|
saved_as: string;
|
|
content_type: string;
|
|
url: string;
|
|
absolute_path: string;
|
|
}
|
|
|
|
export interface AmbiguityCandidate {
|
|
amount: number;
|
|
balance: number;
|
|
}
|
|
|
|
export interface PendingAmbiguity {
|
|
id: string;
|
|
fetch_request: string;
|
|
step_index?: number;
|
|
line: string;
|
|
ocr_amount: number;
|
|
ocr_balance: number;
|
|
prev_balance: number;
|
|
candidates: AmbiguityCandidate[];
|
|
chosen?: AmbiguityCandidate | null;
|
|
resolved_at?: string | null;
|
|
status: "pending" | "resolved";
|
|
created_at: string;
|
|
}
|
|
|
|
export interface ResolveAmbiguityPayload {
|
|
chosen: {
|
|
amount: number;
|
|
balance: number;
|
|
};
|
|
}
|
|
|
|
export type SSEEventStep =
|
|
| "load_content" | "raw_lines" | "txn_blocks" | "txn_dicts"
|
|
| "resume_extract" | "extract" | "paused" | "complete" | "enrich"
|
|
| "save_expenses" | "pipeline";
|
|
|
|
export type SSEEventStatus =
|
|
| "started" | "completed" | "skipped" | "paused" | "progress" | "failed";
|
|
|
|
export interface ProgressMessage {
|
|
lines?: number;
|
|
blocks?: number;
|
|
count?: number;
|
|
unit?: string;
|
|
raw_ocr_line?: string;
|
|
error?: string;
|
|
}
|
|
|
|
export interface SSEEvent {
|
|
step: SSEEventStep;
|
|
status: SSEEventStatus;
|
|
message: ProgressMessage;
|
|
}
|
|
|
|
export interface FetchRequestFilters {
|
|
status?: FetchRequestStatus[];
|
|
account_name?: string;
|
|
source_type?: "file" | "email";
|
|
}
|
|
|
|
export function formatApiError(err: any): string {
|
|
if (!err?.response) return err?.message || "Request failed";
|
|
const data = err.response.data;
|
|
const status = err.response.status;
|
|
|
|
if (status === 422 && Array.isArray(data?.detail)) {
|
|
return data.detail.map((d: any) => {
|
|
const field = d.loc?.filter((s: string) => s !== "body").pop() || "field";
|
|
if (d.type === "value_error.missing") return `Missing: ${field}`;
|
|
return `${field}: ${d.msg}`;
|
|
}).join("; ");
|
|
}
|
|
|
|
if (typeof data?.detail === "string") return data.detail;
|
|
return `Request failed (${status})`;
|
|
}
|
|
|
|
export const RETRY_MAX = 3;
|