3 Commits

Author SHA1 Message Date
7c33bd9c7c enumOptions and enum reader 2026-06-04 03:37:44 +05:30
2dbe9a5270 icici not icici_ocr 2026-06-03 19:01:38 +05:30
5cf2a4c3c4 Fetch Request UI (#9)
## Summary

Add Fetch Request pipeline UI with real-time SSE progress tracking, ambiguity resolution, enhanced list page with filtering/retry, and detail page with stepper/event feed. Includes new API primitives (`api.patch`, `usePatch` hook) and extensive type definitions for SSE events, ambiguity resolution, and pipeline statuses.

## Changes by file

### `react-openapi/api/client.ts`
- Added `api.patch()` method (delegates to `AxiosInstance.patch`)

### `react-openapi/hooks/useResource.ts`
- Added `usePatch()` mutation hook — sends PATCH `/{endpoint}/{id}` with partial data, invalidates list + detail query caches on success

### `src/FetchRequestDetail.tsx` **(new, +675 lines)**
Full detail page for a single fetch request, composed of:

- **Header** — status chip, account name, source type (File/Email), date range, created/completed timestamps
- **Progress bar** — `LinearProgress` with percentage computed via `computeProgressPercent()`:
  - Extract phase: snaps to 10% on first `raw_lines`/`txn_blocks` event
  - Raw Expense: ratio of `displayParsedCount` / `txnBlockCount` × 20%
  - Enrich: ratio of `stepStats.enrich_count` / `txnDictCount` × 50%
  - Save: ratio of `stepStats.save_count` / `txnDictCount` × 20%
- **Retry section** — retry count bar (`retryCount/RETRY_MAX`) + "Retry" button if failed & not exhausted
- **Error/success alerts** — `error_message` display, max-retries info
- **Pipeline Stepper** — 4-step (`Extract` → `Raw Expense` → `Enrich` → `Save`) with custom icons per state: completed (CheckCircle, green), active (CircularProgress, animating), paused (WarningAmber, amber), failed (ErrorIcon, red), inactive (step number, grey). Step count labels shown below each label (e.g. `150/246`, `100/246`)
- **Progress Events feed** — auto-scrolling list of SSE events with deduplication:
  - Only latest `progress` event per step (`txn_dicts`, `enrich`, `save_expenses`)
  - `started` events hidden when a terminal event (`completed`/`skipped`/`paused`/`failed`) follows
  - `load_content` events excluded entirely
  - Connection status indicator (green dot / red dot)
- **Ambiguity Resolution** — When pipeline is `paused`, shows ambiguity cards:
  - Raw line in monospace code block
  - OCR amount/balance (struck through), previous balance
  - Candidate buttons with credit/debit coloring (green for positive, red for negative)
  - Resolved state shows green alert with chosen values
  - "All resolved" vs "Pipeline paused" alerts
- **SSE connection** — `EventSource` to `{baseUrl}/fetch-requests/{id}/events`:
  - Tracks `progress`, `completed`, `paused`, `failed` events
  - On `paused`: refetch request + ambiguities
  - On `failed`: refetch request + show error snackbar from `message.error`
  - On `completed`/`resume_extract`: refetch request
  - Cleans up on unmount
- **Snackbar** — pipeline failure notification (6s, bottom-center)

### `src/FetchRequests.tsx` **(+347/−73 lines)**
Major enhancement to the list page:

- **New filter bar** (replaces plain list header):
  - Status multi-select (pending, processing, paused, raw_expenses_done, enriched_done, completed, failed)
  - Account name text filter
  - Source type toggle (All / File / Email)
  - Refresh button
- **Account autocomplete** — fetches accounts list via `useResourceByName("accounts")`, provides dropdown
- **Format dropdown** — driven by `resourceOverrides` config (`fetchRes.fields.source.schema.format.options`), fallback `["axis", "icici_ocr"]`
- **Date pickers** — changed from `datetime-local` to `date`, capped at today's date
- **Navigation** — on create, navigates to `/fetch-requests/{id}` via `useNavigate`
- **Row actions**: retry (failed, retry_count < 3), navigate to detail (paused), delete with confirmation dialog
- **Copy fingerprint** — icon button copies to clipboard with snackbar confirmation
- **Sorting** — table sorted by `created_at` descending
- **Table columns** — changed from `[Fingerprint, Source, Account, Status, Created, Actions]` to `[ID, Account, Source, Date Range, Status, Retries, Created, Actions]`
- **Retry count display** — shows `retry_count/RETRY_MAX` when >0, otherwise `—`
- **Status tooltip** — shows `error_message` on hover when present
- **Status icons** — new `statusIcons` map: ScheduleIcon (pending), CircularProgress (processing), WarningAmber (paused), HourglassEmpty (raw_expenses_done/enriched_done), CheckCircle (completed), ErrorIcon (failed)
- **Error handling** — 409 conflict detection (duplicate fingerprint), 422 validation via `formatApiError()`
- **`handleRetry`** — PATCH `{status: "pending"}` on failed requests, success/error snackbar

### `src/features/fetch-requests/fetch-requests.models.ts` **(+97 lines)**
New types and helpers:

- Added `"paused"` to `FetchRequestStatus`
- `FileSource`: added `raw_lines`, `txn_blocks`, `txn_dicts`, `txn_dict_count`, `txn_dicts_count`
- `EmailSource`: added `txn_dict_count`, `txn_dicts_count`
- `FetchRequest` added `retry_count`
- **New interfaces**: `FetchRequestUpdate`, `AmbiguityCandidate`, `PendingAmbiguity`, `ResolveAmbiguityPayload`, `FetchRequestFilters`
- **SSE types**:
  - `SSEEventStep`: `load_content | raw_lines | txn_blocks | txn_dicts | resume_extract | extract | paused | complete | enrich | save_expenses | pipeline`
  - `SSEEventStatus`: `started | completed | skipped | paused | progress | failed`
  - `ProgressMessage`: `lines? | blocks? | count? | unit? | raw_ocr_line? | error?`
  - `SSEEvent: { step, status, message }`
- **Helpers**: `formatApiError()` — parses FastAPI 422 validation detail arrays (`"Missing: field_name"`), `RETRY_MAX = 3`

### `src/features/fetch-requests/index.ts` **(+13 lines)**
Barrel exports for all new types (`FetchRequestUpdate`, `FetchRequestFilters`, `PendingAmbiguity`, `AmbiguityCandidate`, `ResolveAmbiguityPayload`, `SSEEvent`, `SSEEventStep`, `SSEEventStatus`, `ProgressMessage`), value exports (`RETRY_MAX`, `formatApiError`), and new hooks (`useUpdateFetchRequest`, `useFetchRequestAmbiguities`, `useResolveAmbiguity`)

### `src/features/fetch-requests/useFetchRequests.ts` **(+49 lines)**
Added hooks:

- `useUpdateFetchRequest()` → `usePatch("fetch-requests")`
- `useFetchRequestAmbiguities(fetchRequestId)` → `useQuery` for `GET /fetch-requests/{id}/ambiguities`
- `useResolveAmbiguity()` → `useMutation` for `POST /ambiguities/{id}/resolve` with cache invalidation of both ambiguities and detail queries

### `src/main.jsx` **(+2 lines)**
- Added import for `FetchRequestDetail`
- Added route `{ path: "/fetch-requests/:id", component: FetchRequestDetail, headerTitle: "Fetch Request" }`

Reviewed-on: #9
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
2026-05-31 09:28:26 +00:00
9 changed files with 110 additions and 30 deletions

View File

@@ -31,6 +31,7 @@ import VisibilityIcon from '@mui/icons-material/Visibility';
import MoreVertIcon from '@mui/icons-material/MoreVert';
import { useNavigate } from 'react-router-dom';
import { ResourceConfig } from '../types/config';
import { getFieldOptions, toGridValueOptions } from '../utils/options';
interface EnhancedTableProps {
config: ResourceConfig;
@@ -96,8 +97,7 @@ export default function EnhancedTable({
}
if (muiType === 'singleSelect' && field.options) {
// @ts-ignore
col.valueOptions = field.options;
col.valueOptions = toGridValueOptions(getFieldOptions(field));
}
return col;
@@ -379,6 +379,11 @@ function FieldRenderer({ params, field, fieldKey, config, onNavigate, navigate,
if (field.type === 'datetime' || field.type === 'date') return value ? new Date(value).toLocaleString() : '';
if (field.type === 'enum' && field.options) {
const opt = getFieldOptions(field).find(o => o.key === value);
return opt?.value ?? value;
}
if (isPk && !isMobile) {
return (
<Chip

View File

@@ -11,6 +11,7 @@ import {
import DoneIcon from "@mui/icons-material/Done";
import FilterListIcon from "@mui/icons-material/FilterList";
import { ResourceField, ResourceMode } from "../types/config";
import { getFieldOptions } from "../utils/options";
function FilterAutocomplete({
options,
@@ -110,7 +111,9 @@ function extractOptions(
): string[] {
const values = new Set<string>();
if (field.options) return field.options;
if (field.type === 'enum' && field.options) {
return getFieldOptions(field).map(o => o.key);
}
if (!data) return [];
const pull = (item: any): string | null => {

View File

@@ -12,6 +12,7 @@ import {
Divider,
} from '@mui/material';
import { ResourceField } from '../../types/config';
import { getFieldOptions } from '../../utils/options';
import ImageUploadField from './ImageUploadField';
interface FormFieldProps {
@@ -73,40 +74,40 @@ export default function FormField({
if (field.relation && relationDataMap[field.relation]) {
const relationData = relationDataMap[field.relation].data;
const isArrayRelation = field.type === 'array';
const options = getFieldOptions(field, relationData);
const keyField = field.enumOption?.key ?? 'id';
// Determine how to display the related item
const getOptionLabel = (option: any) => {
if (!option) return "";
if (field.displayField && option[field.displayField]) return option[field.displayField];
// Standard naming fields
return option.name || option.title || option.label || option.id || JSON.stringify(option);
};
const getOptionValue = (option: any) => {
// Return the whole object to maintain identity
return option;
};
// Normalize value: API returns whole objects on GET, but form uses key strings
const normalizedValue = (() => {
if (isArrayRelation && Array.isArray(value)) {
return value.map((v: any) => (v != null && typeof v === 'object' ? String(v[keyField] ?? '') : String(v)));
}
if (value != null && typeof value === 'object') {
return String(value[keyField] ?? '');
}
return value ?? (isArrayRelation ? [] : "");
})();
return (
<FormControl fullWidth>
<InputLabel shrink>{label}</InputLabel>
<Select
multiple={isArrayRelation}
value={value || (isArrayRelation ? [] : "")}
value={normalizedValue}
label={label}
displayEmpty
onChange={(e) => onChange(e.target.value)}
disabled={disabled}
renderValue={(selected: any) => {
if (isArrayRelation) {
return (selected as any[]).map(getOptionLabel).join(', ');
return (selected as string[]).map(k => options.find(o => o.key === k)?.value ?? k).join(', ');
}
return getOptionLabel(selected);
return options.find(o => o.key === selected)?.value ?? selected;
}}
>
{relationData.map((option) => (
<MenuItem key={option.id || JSON.stringify(option)} value={getOptionValue(option)}>
{getOptionLabel(option)}
{options.map((opt) => (
<MenuItem key={opt.key} value={opt.key}>
{opt.value}
</MenuItem>
))}
</Select>
@@ -149,6 +150,7 @@ export default function FormField({
// 5. Enum Handling
if (field.type === 'enum' && field.options) {
const options = getFieldOptions(field);
return (
<FormControl fullWidth>
<InputLabel>{label}</InputLabel>
@@ -158,9 +160,9 @@ export default function FormField({
onChange={(e) => onChange(e.target.value)}
disabled={disabled}
>
{field.options.map((opt: string) => (
<MenuItem key={opt} value={opt}>
{opt}
{options.map((opt) => (
<MenuItem key={opt.key} value={opt.key}>
{opt.value}
</MenuItem>
))}
</Select>

View File

@@ -10,6 +10,16 @@ export type FieldType =
| 'object'
| 'array';
export interface SelectOption {
key: string;
value: string;
}
export interface EnumOption {
key: string;
value: string | string[];
}
export interface ResourceField {
type: FieldType;
label: string;
@@ -19,8 +29,10 @@ export interface ResourceField {
schema?: Record<string, ResourceField>;
displayField?: string | string[];
formatter?: (value: any) => string;
relation?: string; // Name of the target resource
relation?: string;
filterType?: "autocomplete" | "multiselect" | "number-range" | "date-range";
enumOption?: EnumOption;
enumLabels?: Record<string, string>;
}
export type ResourceMode = "server" | "client";
@@ -38,6 +50,7 @@ export interface ResourceConfig {
mode?: ResourceMode;
fields?: string[];
};
enumOption?: EnumOption;
}
export interface AppConfig {

View File

@@ -1,13 +1,14 @@
/**
* This file contains application-specific overrides and configuration
* for the generic Admin Panel.
*/
export interface EnumOption {
key: string;
value: string | string[];
}
export interface FieldOverride {
displayField?: string | string[];
display?: boolean;
formatter?: (value: any) => string;
filterType?: "autocomplete" | "multiselect" | "number-range" | "date-range";
enumLabels?: Record<string, string>;
}
export interface ResourceOverride {
@@ -18,4 +19,5 @@ export interface ResourceOverride {
mode?: "server" | "client";
fields?: string[];
};
enumOption?: EnumOption;
}

View File

@@ -80,6 +80,21 @@ function parseSchemaFields(
const relation = schemaToResourceMap.get(targetSchema);
if (relation) {
fields[key].relation = relation;
// Propagate enumOption from target resource config, or derive from target schema
const explicitEnumOption = configuration[relation]?.enumOption;
if (explicitEnumOption) {
fields[key].enumOption = explicitEnumOption;
} else {
const targetProps = targetSchema.properties || {};
const valueField = Object.entries(targetProps).find(
([name, p]: [string, any]) => name !== 'id' && p.type === 'string'
)?.[0];
fields[key].enumOption = {
key: 'id',
value: valueField ?? 'id',
};
}
}
// Recursively parse nested objects (only if not a relation)

View File

@@ -0,0 +1,28 @@
import { ResourceField, SelectOption } from "../types/config";
export function getFieldOptions(field: ResourceField, relationData?: any[]): SelectOption[] {
if (field.type === 'enum' && field.options) {
return field.options.map(opt => ({
key: opt,
value: field.enumLabels?.[opt] ?? opt,
}));
}
if (field.relation) {
const data = relationData ?? [];
const enumOption = field.enumOption ?? { key: 'id', value: 'name' };
return data.map(item => ({
key: String(item[enumOption.key] ?? ''),
value: Array.isArray(enumOption.value)
? enumOption.value.map(k => item[k]).filter(v => v != null).join(' ')
: String(item[enumOption.value] ?? ''),
}));
}
return [];
}
export function toGridValueOptions(options: SelectOption[]): { value: string; label: string }[] {
return options.map(opt => ({ value: opt.key, label: opt.value }));
}

View File

@@ -129,7 +129,7 @@ export default function FetchRequests() {
const config = useConfig();
const fetchRes = config?.resources.find((r: any) => r.name === "fetch-requests");
const formatOptions: string[] = (fetchRes?.fields?.source?.schema?.format?.options as string[]) ?? ["axis", "icici_ocr"];
const formatOptions: string[] = (fetchRes?.fields?.source?.schema?.format?.options as string[]) ?? ["axis", "icici"];
const createMutation = useCreateFetchRequest();
const updateMutation = useUpdateFetchRequest();

View File

@@ -50,6 +50,18 @@ export const configuration: Record<string, ResourceOverride> = {
}
},
},
accounts: {
enumOption: {
key: 'id',
value: ['name', 'number']
}
},
tags: {
enumOption: {
key: 'id',
value: ['icon', 'name']
}
},
};
export const profileConfiguration = {