apps/khata-ui
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fetch Request UI (#9)

Browse Source 
## Summary

Add Fetch Request pipeline UI with real-time SSE progress tracking, ambiguity resolution, enhanced list page with filtering/retry, and detail page with stepper/event feed. Includes new API primitives (`api.patch`, `usePatch` hook) and extensive type definitions for SSE events, ambiguity resolution, and pipeline statuses.

## Changes by file

### `react-openapi/api/client.ts`
- Added `api.patch()` method (delegates to `AxiosInstance.patch`)

### `react-openapi/hooks/useResource.ts`
- Added `usePatch()` mutation hook — sends PATCH `/{endpoint}/{id}` with partial data, invalidates list + detail query caches on success

### `src/FetchRequestDetail.tsx` **(new, +675 lines)**
Full detail page for a single fetch request, composed of:

- **Header** — status chip, account name, source type (File/Email), date range, created/completed timestamps
- **Progress bar** — `LinearProgress` with percentage computed via `computeProgressPercent()`:
  - Extract phase: snaps to 10% on first `raw_lines`/`txn_blocks` event
  - Raw Expense: ratio of `displayParsedCount` / `txnBlockCount` × 20%
  - Enrich: ratio of `stepStats.enrich_count` / `txnDictCount` × 50%
  - Save: ratio of `stepStats.save_count` / `txnDictCount` × 20%
- **Retry section** — retry count bar (`retryCount/RETRY_MAX`) + "Retry" button if failed & not exhausted
- **Error/success alerts** — `error_message` display, max-retries info
- **Pipeline Stepper** — 4-step (`Extract` → `Raw Expense` → `Enrich` → `Save`) with custom icons per state: completed (CheckCircle, green), active (CircularProgress, animating), paused (WarningAmber, amber), failed (ErrorIcon, red), inactive (step number, grey). Step count labels shown below each label (e.g. `150/246`, `100/246`)
- **Progress Events feed** — auto-scrolling list of SSE events with deduplication:
  - Only latest `progress` event per step (`txn_dicts`, `enrich`, `save_expenses`)
  - `started` events hidden when a terminal event (`completed`/`skipped`/`paused`/`failed`) follows
  - `load_content` events excluded entirely
  - Connection status indicator (green dot / red dot)
- **Ambiguity Resolution** — When pipeline is `paused`, shows ambiguity cards:
  - Raw line in monospace code block
  - OCR amount/balance (struck through), previous balance
  - Candidate buttons with credit/debit coloring (green for positive, red for negative)
  - Resolved state shows green alert with chosen values
  - "All resolved" vs "Pipeline paused" alerts
- **SSE connection** — `EventSource` to `{baseUrl}/fetch-requests/{id}/events`:
  - Tracks `progress`, `completed`, `paused`, `failed` events
  - On `paused`: refetch request + ambiguities
  - On `failed`: refetch request + show error snackbar from `message.error`
  - On `completed`/`resume_extract`: refetch request
  - Cleans up on unmount
- **Snackbar** — pipeline failure notification (6s, bottom-center)

### `src/FetchRequests.tsx` **(+347/−73 lines)**
Major enhancement to the list page:

- **New filter bar** (replaces plain list header):
  - Status multi-select (pending, processing, paused, raw_expenses_done, enriched_done, completed, failed)
  - Account name text filter
  - Source type toggle (All / File / Email)
  - Refresh button
- **Account autocomplete** — fetches accounts list via `useResourceByName("accounts")`, provides dropdown
- **Format dropdown** — driven by `resourceOverrides` config (`fetchRes.fields.source.schema.format.options`), fallback `["axis", "icici_ocr"]`
- **Date pickers** — changed from `datetime-local` to `date`, capped at today's date
- **Navigation** — on create, navigates to `/fetch-requests/{id}` via `useNavigate`
- **Row actions**: retry (failed, retry_count < 3), navigate to detail (paused), delete with confirmation dialog
- **Copy fingerprint** — icon button copies to clipboard with snackbar confirmation
- **Sorting** — table sorted by `created_at` descending
- **Table columns** — changed from `[Fingerprint, Source, Account, Status, Created, Actions]` to `[ID, Account, Source, Date Range, Status, Retries, Created, Actions]`
- **Retry count display** — shows `retry_count/RETRY_MAX` when >0, otherwise `—`
- **Status tooltip** — shows `error_message` on hover when present
- **Status icons** — new `statusIcons` map: ScheduleIcon (pending), CircularProgress (processing), WarningAmber (paused), HourglassEmpty (raw_expenses_done/enriched_done), CheckCircle (completed), ErrorIcon (failed)
- **Error handling** — 409 conflict detection (duplicate fingerprint), 422 validation via `formatApiError()`
- **`handleRetry`** — PATCH `{status: "pending"}` on failed requests, success/error snackbar

### `src/features/fetch-requests/fetch-requests.models.ts` **(+97 lines)**
New types and helpers:

- Added `"paused"` to `FetchRequestStatus`
- `FileSource`: added `raw_lines`, `txn_blocks`, `txn_dicts`, `txn_dict_count`, `txn_dicts_count`
- `EmailSource`: added `txn_dict_count`, `txn_dicts_count`
- `FetchRequest` added `retry_count`
- **New interfaces**: `FetchRequestUpdate`, `AmbiguityCandidate`, `PendingAmbiguity`, `ResolveAmbiguityPayload`, `FetchRequestFilters`
- **SSE types**:
  - `SSEEventStep`: `load_content | raw_lines | txn_blocks | txn_dicts | resume_extract | extract | paused | complete | enrich | save_expenses | pipeline`
  - `SSEEventStatus`: `started | completed | skipped | paused | progress | failed`
  - `ProgressMessage`: `lines? | blocks? | count? | unit? | raw_ocr_line? | error?`
  - `SSEEvent: { step, status, message }`
- **Helpers**: `formatApiError()` — parses FastAPI 422 validation detail arrays (`"Missing: field_name"`), `RETRY_MAX = 3`

### `src/features/fetch-requests/index.ts` **(+13 lines)**
Barrel exports for all new types (`FetchRequestUpdate`, `FetchRequestFilters`, `PendingAmbiguity`, `AmbiguityCandidate`, `ResolveAmbiguityPayload`, `SSEEvent`, `SSEEventStep`, `SSEEventStatus`, `ProgressMessage`), value exports (`RETRY_MAX`, `formatApiError`), and new hooks (`useUpdateFetchRequest`, `useFetchRequestAmbiguities`, `useResolveAmbiguity`)

### `src/features/fetch-requests/useFetchRequests.ts` **(+49 lines)**
Added hooks:

- `useUpdateFetchRequest()` → `usePatch("fetch-requests")`
- `useFetchRequestAmbiguities(fetchRequestId)` → `useQuery` for `GET /fetch-requests/{id}/ambiguities`
- `useResolveAmbiguity()` → `useMutation` for `POST /ambiguities/{id}/resolve` with cache invalidation of both ambiguities and detail queries

### `src/main.jsx` **(+2 lines)**
- Added import for `FetchRequestDetail`
- Added route `{ path: "/fetch-requests/:id", component: FetchRequestDetail, headerTitle: "Fetch Request" }`

Reviewed-on: #9
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-05-31 09:28:26 +00:00
committed by aetos
parent d4a79c785d
commit 5cf2a4c3c4
8 changed files with 1132 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

347
src/FetchRequests.tsx
View File

@@ -24,14 +24,28 @@ import {
DialogContent,
DialogContentText,
DialogActions,
Tooltip,
Select,
MenuItem,
InputLabel,
FormControl,
OutlinedInput,
Autocomplete,
} from "@mui/material";
import DeleteIcon from "@mui/icons-material/Delete";
import CloudUploadIcon from "@mui/icons-material/CloudUpload";
import RefreshIcon from "@mui/icons-material/Refresh";
import ContentCopyIcon from "@mui/icons-material/ContentCopy";
import ReplayIcon from "@mui/icons-material/Replay";
import CheckCircleIcon from "@mui/icons-material/CheckCircle";
import ErrorIcon from "@mui/icons-material/Error";
import ScheduleIcon from "@mui/icons-material/Schedule";
import HourglassEmptyIcon from "@mui/icons-material/HourglassEmpty";
import WarningAmberIcon from "@mui/icons-material/WarningAmber";
import {
useFetchRequestsList,
useCreateFetchRequest,
useUpdateFetchRequest,
useDeleteFetchRequest,
useUploadFile,
} from "./features/fetch-requests";
@@ -41,22 +55,49 @@ import type {
FileSource,
EmailSource,
} from "./features/fetch-requests";
import { RETRY_MAX, formatApiError } from "./features/fetch-requests";
import { useNavigate } from "react-router-dom";
import { useResourceByName, useConfig } from "../react-openapi";
const statusColors: Record<FetchRequestStatus, "default" | "primary" | "warning" | "info" | "success" | "error"> = {
pending: "default",
processing: "info",
paused: "warning",
raw_expenses_done: "primary",
enriched_done: "warning",
completed: "success",
failed: "error",
};
const statusIcons: Record<FetchRequestStatus, React.ReactNode> = {
pending: <ScheduleIcon sx={{ fontSize: 16 }} />,
processing: <CircularProgress size={14} sx={{ mr: 0.5 }} />,
paused: <WarningAmberIcon sx={{ fontSize: 16, color: "warning.main" }} />,
raw_expenses_done: <HourglassEmptyIcon sx={{ fontSize: 16 }} />,
enriched_done: <HourglassEmptyIcon sx={{ fontSize: 16 }} />,
completed: <CheckCircleIcon sx={{ fontSize: 16, color: "success.main" }} />,
failed: <ErrorIcon sx={{ fontSize: 16, color: "error.main" }} />,
};
function formatDate(iso: string) {
const d = new Date(iso);
return d.toLocaleString();
}
function formatDateRange(start?: string, end?: string) {
if (!start && !end) return "—";
const s = start ? new Date(start).toLocaleDateString() : "?";
const e = end ? new Date(end).toLocaleDateString() : "?";
return `${s} → ${e}`;
}
function shortId(fp: string) {
return fp.length > 8 ? fp.slice(0, 8) + "…" : fp;
}
export default function FetchRequests() {
const navigate = useNavigate();
const [sourceType, setSourceType] = React.useState<"file" | "email">("file");
const [accountName, setAccountName] = React.useState("");
const [payorUsername, setPayorUsername] = React.useState("aetos");
@@ -71,8 +112,27 @@ export default function FetchRequests() {
const [snackbar, setSnackbar] = React.useState<{ message: string; severity: "success" | "error" } | null>(null);
const [deleteTarget, setDeleteTarget] = React.useState<FetchRequest | null>(null);
const { data: listData, isLoading, isFetching, refetch } = useFetchRequestsList();
const [statusFilter, setStatusFilter] = React.useState<string[]>([]);
const [accountFilter, setAccountFilter] = React.useState("");
const [sourceFilter, setSourceFilter] = React.useState<"all" | "file" | "email">("all");
const { data: listData, isLoading, isFetching, refetch } = useFetchRequestsList({
...(statusFilter.length > 0 ? { status: statusFilter.join(",") } : {}),
...(accountFilter ? { account_name: accountFilter } : {}),
...(sourceFilter !== "all" ? { source_type: sourceFilter } : {}),
});
const { useList: useAccountsList } = useResourceByName("accounts");
const { data: accountsData } = useAccountsList();
const accountOptions: string[] = React.useMemo(() => {
return (accountsData?.data ?? []).map((a: any) => a.name).filter(Boolean);
}, [accountsData]);
const config = useConfig();
const fetchRes = config?.resources.find((r: any) => r.name === "fetch-requests");
const formatOptions: string[] = (fetchRes?.fields?.source?.schema?.format?.options as string[]) ?? ["axis", "icici_ocr"];
const createMutation = useCreateFetchRequest();
const updateMutation = useUpdateFetchRequest();
const deleteMutation = useDeleteFetchRequest();
const uploadMutation = useUploadFile();
@@ -106,7 +166,7 @@ export default function FetchRequests() {
}
try {
await createMutation.mutateAsync({
const result = await createMutation.mutateAsync({
source,
account_name: accountName,
payor_username: payorUsername,
@@ -115,8 +175,13 @@ export default function FetchRequests() {
});
setSnackbar({ message: "Fetch request created", severity: "success" });
resetForm();
navigate(`/fetch-requests/${result.id}`);
} catch (err: any) {
setSnackbar({ message: err?.response?.data?.detail || "Failed to create fetch request", severity: "error" });
if (err?.response?.status === 409) {
setSnackbar({ message: "Duplicate — same fingerprint already exists", severity: "error" });
} else {
setSnackbar({ message: formatApiError(err) || "Failed to create fetch request", severity: "error" });
}
}
};
@@ -132,6 +197,15 @@ export default function FetchRequests() {
setEndDate("");
};
const handleRetry = async (req: FetchRequest) => {
try {
await updateMutation.mutateAsync({ id: req.id, data: { status: "pending" } });
setSnackbar({ message: "Retrying fetch request", severity: "success" });
} catch {
setSnackbar({ message: "Failed to retry", severity: "error" });
}
};
const handleDelete = async () => {
if (!deleteTarget) return;
try {
@@ -143,6 +217,8 @@ export default function FetchRequests() {
setDeleteTarget(null);
};
const sourceTypeOptions: ("all" | "file" | "email")[] = ["all", "file", "email"];
return (
<Container sx={{ mt: 4, mb: 4 }}>
<Typography variant="h5" fontWeight="bold" gutterBottom>
@@ -189,37 +265,61 @@ export default function FetchRequests() {
Uploaded as: {uploadedPath}
</Alert>
)}
<TextField label="Format (csv, pdf, ...)" value={format} onChange={(e) => setFormat(e.target.value)} size="small" />
<FormControl size="small">
<InputLabel>Format</InputLabel>
<Select value={format} onChange={(e) => setFormat(e.target.value)} label="Format">
{formatOptions.map((opt) => (
<MenuItem key={opt} value={opt}>{opt}</MenuItem>
))}
</Select>
</FormControl>
</>
) : (
<>
<TextField label="Format" value={format} onChange={(e) => setFormat(e.target.value)} size="small" helperText="e.g. email, pdf, csv" />
<FormControl size="small">
<InputLabel>Format</InputLabel>
<Select value={format} onChange={(e) => setFormat(e.target.value)} label="Format">
{formatOptions.map((opt) => (
<MenuItem key={opt} value={opt}>{opt}</MenuItem>
))}
</Select>
</FormControl>
<TextField label="From Email" value={fromEmail} onChange={(e) => setFromEmail(e.target.value)} size="small" />
<TextField label="Subject" value={subject} onChange={(e) => setSubject(e.target.value)} size="small" />
<TextField label="Raw Terms" value={rawTerms} onChange={(e) => setRawTerms(e.target.value)} size="small" helperText="Comma-separated search terms" />
</>
)}
<TextField label="Account Name" value={accountName} onChange={(e) => setAccountName(e.target.value)} size="small" required />
<Autocomplete
options={accountOptions}
value={accountName || null}
onChange={(_, val) => setAccountName(val ?? "")}
renderInput={(params) => (
<TextField {...params} label="Account Name" size="small" required />
)}
sx={{ "& .MuiOutlinedInput-root": { height: "auto", minHeight: "2.5rem" } }}
/>
<TextField label="Payor Username" value={payorUsername} onChange={(e) => setPayorUsername(e.target.value)} size="small" helperText="Default: aetos" />
<Box sx={{ display: "flex", gap: 2 }}>
<TextField
label="Start Date"
type="datetime-local"
type="date"
value={startDate}
onChange={(e) => setStartDate(e.target.value)}
size="small"
InputLabelProps={{ shrink: true }}
inputProps={{ max: new Date().toISOString().split("T")[0] }}
sx={{ flex: 1 }}
/>
<TextField
label="End Date"
type="datetime-local"
type="date"
value={endDate}
onChange={(e) => setEndDate(e.target.value)}
size="small"
InputLabelProps={{ shrink: true }}
inputProps={{ max: new Date().toISOString().split("T")[0] }}
sx={{ flex: 1 }}
/>
</Box>
@@ -234,79 +334,184 @@ export default function FetchRequests() {
</Box>
</Paper>
<Paper sx={{ borderRadius: 4 }} variant="outlined">
<Box sx={{ display: "flex", alignItems: "center", justifyContent: "space-between", p: 2, pb: 0 }}>
<Typography variant="subtitle1" fontWeight={600}>
Fetch Requests
</Typography>
<Paper sx={{ borderRadius: 4, mb: 2, p: 2 }} variant="outlined">
<Box sx={{ display: "flex", alignItems: "center", gap: 2, flexWrap: "wrap" }}>
<FormControl size="small" sx={{ minWidth: 200 }}>
<InputLabel>Status</InputLabel>
<Select
multiple
value={statusFilter}
onChange={(e) => setStatusFilter(e.target.value as string[])}
input={<OutlinedInput label="Status" />}
renderValue={(selected) => (selected as string[]).join(", ")}
>
{["pending", "processing", "paused", "raw_expenses_done", "enriched_done", "completed", "failed"].map((s) => (
<MenuItem key={s} value={s}>{s.replace(/_/g, " ")}</MenuItem>
))}
</Select>
</FormControl>
<TextField
label="Account"
value={accountFilter}
onChange={(e) => setAccountFilter(e.target.value)}
size="small"
sx={{ minWidth: 160 }}
/>
<ToggleButtonGroup
value={sourceFilter}
exclusive
onChange={(_, val) => val && setSourceFilter(val)}
size="small"
>
{sourceTypeOptions.map((opt) => (
<ToggleButton key={opt} value={opt}>
{opt === "all" ? "All" : opt === "file" ? "File" : "Email"}
</ToggleButton>
))}
</ToggleButtonGroup>
<Box sx={{ flex: 1 }} />
<IconButton onClick={() => refetch()} disabled={isFetching}>
<RefreshIcon />
</IconButton>
</Box>
</Paper>
{isLoading ? (
<Box sx={{ display: "flex", justifyContent: "center", p: 4 }}>
<CircularProgress />
</Box>
) : requests.length === 0 ? (
<Box sx={{ p: 4, textAlign: "center", color: "text.secondary" }}>
No fetch requests yet
</Box>
) : (
<TableContainer>
<Table size="small">
<TableHead>
<TableRow>
<TableCell>Fingerprint</TableCell>
<TableCell>Source</TableCell>
<TableCell>Account</TableCell>
<TableCell>Status</TableCell>
<TableCell>Created</TableCell>
<TableCell align="right">Actions</TableCell>
</TableRow>
</TableHead>
<TableBody>
{requests.map((req: FetchRequest) => (
<TableRow key={req.id}>
<TableCell sx={{ fontFamily: "monospace", fontSize: "0.8rem" }}>
<Box sx={{ display: "flex", alignItems: "center", gap: 0.5 }}>
{req.fingerprint}
{isLoading ? (
<Box sx={{ display: "flex", justifyContent: "center", p: 4 }}>
<CircularProgress />
</Box>
) : requests.length === 0 ? (
<Box sx={{ p: 4, textAlign: "center", color: "text.secondary" }}>
No fetch requests yet
</Box>
) : (
<TableContainer component={Paper} variant="outlined" sx={{ borderRadius: 4 }}>
<Table size="small">
<TableHead>
<TableRow>
<TableCell>ID</TableCell>
<TableCell>Account</TableCell>
<TableCell>Source</TableCell>
<TableCell>Date Range</TableCell>
<TableCell>Status</TableCell>
<TableCell>Retries</TableCell>
<TableCell>Created</TableCell>
<TableCell align="right">Actions</TableCell>
</TableRow>
</TableHead>
<TableBody>
{[...requests]
.sort((a, b) => new Date(b.created_at).getTime() - new Date(a.created_at).getTime())
.map((req: FetchRequest) => (
<TableRow
key={req.id}
hover
onClick={() => navigate(`/fetch-requests/${req.id}`)}
sx={{ cursor: "pointer", "&:last-child td": { border: 0 } }}
>
<TableCell sx={{ fontFamily: "monospace", fontSize: "0.8rem" }}>
<Box sx={{ display: "flex", alignItems: "center", gap: 0.5 }}>
{shortId(req.fingerprint)}
<IconButton
size="small"
onClick={(e) => {
e.stopPropagation();
navigator.clipboard.writeText(req.fingerprint);
setSnackbar({ message: "Copied!", severity: "success" });
}}
sx={{ opacity: 0.5, '&:hover': { opacity: 1 } }}
>
<ContentCopyIcon sx={{ fontSize: 14 }} />
</IconButton>
</Box>
</TableCell>
<TableCell>{req.account_name}</TableCell>
<TableCell>
<Chip
label={"path" in req.source ? "File" : "Email"}
size="small"
variant="outlined"
color={"path" in req.source ? "primary" : "secondary"}
/>
</TableCell>
<TableCell>
<Typography variant="body2" sx={{ fontSize: "0.8rem", whiteSpace: "nowrap" }}>
{formatDateRange((req as any).start_date, (req as any).end_date)}
</Typography>
</TableCell>
<TableCell>
<Box sx={{ display: "flex", alignItems: "center", gap: 0.5 }}>
<Tooltip title={req.error_message || req.status.replace(/_/g, " ")}>
<Chip
icon={statusIcons[req.status] as any}
label={req.status.replace(/_/g, " ")}
color={statusColors[req.status]}
size="small"
/>
</Tooltip>
</Box>
</TableCell>
<TableCell>
{(req.retry_count ?? 0) > 0 ? (
<Typography variant="body2" sx={{ fontSize: "0.8rem" }}>
{req.retry_count}/{RETRY_MAX}
</Typography>
) : (
<Typography variant="body2" sx={{ fontSize: "0.8rem", color: "text.disabled" }}>
—
</Typography>
)}
</TableCell>
<TableCell sx={{ whiteSpace: "nowrap", fontSize: "0.8rem" }}>
{formatDate(req.created_at)}
</TableCell>
<TableCell align="right">
<Box sx={{ display: "flex", gap: 0.5, justifyContent: "flex-end" }}>
{req.status === "paused" && (
<Tooltip title="Resolve ambiguities">
<IconButton
size="small"
onClick={(e) => {
e.stopPropagation();
navigate(`/fetch-requests/${req.id}`);
}}
>
<WarningAmberIcon fontSize="small" color="warning" />
</IconButton>
</Tooltip>
)}
{req.status === "failed" && (req.retry_count ?? 0) < RETRY_MAX && (
<Tooltip title="Retry">
<IconButton
size="small"
onClick={(e) => {
e.stopPropagation();
handleRetry(req);
}}
>
<ReplayIcon fontSize="small" />
</IconButton>
</Tooltip>
)}
<Tooltip title="Delete">
<IconButton
size="small"
onClick={() => {
navigator.clipboard.writeText(req.fingerprint);
setSnackbar({ message: "Copied!", severity: "success" });
onClick={(e) => {
e.stopPropagation();
setDeleteTarget(req);
}}
sx={{ opacity: 0.5, '&:hover': { opacity: 1 } }}
>
<ContentCopyIcon sx={{ fontSize: 14 }} />
<DeleteIcon fontSize="small" />
</IconButton>
</Box>
</TableCell>
<TableCell>
{"path" in req.source ? "File" : "Email"}
</TableCell>
<TableCell>{req.account_name}</TableCell>
<TableCell>
<Chip
label={req.status.replace(/_/g, " ")}
color={statusColors[req.status]}
size="small"
/>
</TableCell>
<TableCell>{formatDate(req.created_at)}</TableCell>
<TableCell align="right">
<IconButton size="small" onClick={() => setDeleteTarget(req)}>
<DeleteIcon fontSize="small" />
</IconButton>
</TableCell>
</TableRow>
))}
</TableBody>
</Table>
</TableContainer>
)}
</Paper>
</Tooltip>
</Box>
</TableCell>
</TableRow>
))}
</TableBody>
</Table>
</TableContainer>
)}
<Snackbar
open={!!snackbar}