a54250b53d6e95ff542cf1bf896966c6f6c0bd9e
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| a54250b53d | common fields | |||
| 5cf2a4c3c4 |
Fetch Request UI (#9)
## Summary
Add Fetch Request pipeline UI with real-time SSE progress tracking, ambiguity resolution, enhanced list page with filtering/retry, and detail page with stepper/event feed. Includes new API primitives (`api.patch`, `usePatch` hook) and extensive type definitions for SSE events, ambiguity resolution, and pipeline statuses.
## Changes by file
### `react-openapi/api/client.ts`
- Added `api.patch()` method (delegates to `AxiosInstance.patch`)
### `react-openapi/hooks/useResource.ts`
- Added `usePatch()` mutation hook — sends PATCH `/{endpoint}/{id}` with partial data, invalidates list + detail query caches on success
### `src/FetchRequestDetail.tsx` **(new, +675 lines)**
Full detail page for a single fetch request, composed of:
- **Header** — status chip, account name, source type (File/Email), date range, created/completed timestamps
- **Progress bar** — `LinearProgress` with percentage computed via `computeProgressPercent()`:
- Extract phase: snaps to 10% on first `raw_lines`/`txn_blocks` event
- Raw Expense: ratio of `displayParsedCount` / `txnBlockCount` × 20%
- Enrich: ratio of `stepStats.enrich_count` / `txnDictCount` × 50%
- Save: ratio of `stepStats.save_count` / `txnDictCount` × 20%
- **Retry section** — retry count bar (`retryCount/RETRY_MAX`) + "Retry" button if failed & not exhausted
- **Error/success alerts** — `error_message` display, max-retries info
- **Pipeline Stepper** — 4-step (`Extract` → `Raw Expense` → `Enrich` → `Save`) with custom icons per state: completed (CheckCircle, green), active (CircularProgress, animating), paused (WarningAmber, amber), failed (ErrorIcon, red), inactive (step number, grey). Step count labels shown below each label (e.g. `150/246`, `100/246`)
- **Progress Events feed** — auto-scrolling list of SSE events with deduplication:
- Only latest `progress` event per step (`txn_dicts`, `enrich`, `save_expenses`)
- `started` events hidden when a terminal event (`completed`/`skipped`/`paused`/`failed`) follows
- `load_content` events excluded entirely
- Connection status indicator (green dot / red dot)
- **Ambiguity Resolution** — When pipeline is `paused`, shows ambiguity cards:
- Raw line in monospace code block
- OCR amount/balance (struck through), previous balance
- Candidate buttons with credit/debit coloring (green for positive, red for negative)
- Resolved state shows green alert with chosen values
- "All resolved" vs "Pipeline paused" alerts
- **SSE connection** — `EventSource` to `{baseUrl}/fetch-requests/{id}/events`:
- Tracks `progress`, `completed`, `paused`, `failed` events
- On `paused`: refetch request + ambiguities
- On `failed`: refetch request + show error snackbar from `message.error`
- On `completed`/`resume_extract`: refetch request
- Cleans up on unmount
- **Snackbar** — pipeline failure notification (6s, bottom-center)
### `src/FetchRequests.tsx` **(+347/−73 lines)**
Major enhancement to the list page:
- **New filter bar** (replaces plain list header):
- Status multi-select (pending, processing, paused, raw_expenses_done, enriched_done, completed, failed)
- Account name text filter
- Source type toggle (All / File / Email)
- Refresh button
- **Account autocomplete** — fetches accounts list via `useResourceByName("accounts")`, provides dropdown
- **Format dropdown** — driven by `resourceOverrides` config (`fetchRes.fields.source.schema.format.options`), fallback `["axis", "icici_ocr"]`
- **Date pickers** — changed from `datetime-local` to `date`, capped at today's date
- **Navigation** — on create, navigates to `/fetch-requests/{id}` via `useNavigate`
- **Row actions**: retry (failed, retry_count < 3), navigate to detail (paused), delete with confirmation dialog
- **Copy fingerprint** — icon button copies to clipboard with snackbar confirmation
- **Sorting** — table sorted by `created_at` descending
- **Table columns** — changed from `[Fingerprint, Source, Account, Status, Created, Actions]` to `[ID, Account, Source, Date Range, Status, Retries, Created, Actions]`
- **Retry count display** — shows `retry_count/RETRY_MAX` when >0, otherwise `—`
- **Status tooltip** — shows `error_message` on hover when present
- **Status icons** — new `statusIcons` map: ScheduleIcon (pending), CircularProgress (processing), WarningAmber (paused), HourglassEmpty (raw_expenses_done/enriched_done), CheckCircle (completed), ErrorIcon (failed)
- **Error handling** — 409 conflict detection (duplicate fingerprint), 422 validation via `formatApiError()`
- **`handleRetry`** — PATCH `{status: "pending"}` on failed requests, success/error snackbar
### `src/features/fetch-requests/fetch-requests.models.ts` **(+97 lines)**
New types and helpers:
- Added `"paused"` to `FetchRequestStatus`
- `FileSource`: added `raw_lines`, `txn_blocks`, `txn_dicts`, `txn_dict_count`, `txn_dicts_count`
- `EmailSource`: added `txn_dict_count`, `txn_dicts_count`
- `FetchRequest` added `retry_count`
- **New interfaces**: `FetchRequestUpdate`, `AmbiguityCandidate`, `PendingAmbiguity`, `ResolveAmbiguityPayload`, `FetchRequestFilters`
- **SSE types**:
- `SSEEventStep`: `load_content | raw_lines | txn_blocks | txn_dicts | resume_extract | extract | paused | complete | enrich | save_expenses | pipeline`
- `SSEEventStatus`: `started | completed | skipped | paused | progress | failed`
- `ProgressMessage`: `lines? | blocks? | count? | unit? | raw_ocr_line? | error?`
- `SSEEvent: { step, status, message }`
- **Helpers**: `formatApiError()` — parses FastAPI 422 validation detail arrays (`"Missing: field_name"`), `RETRY_MAX = 3`
### `src/features/fetch-requests/index.ts` **(+13 lines)**
Barrel exports for all new types (`FetchRequestUpdate`, `FetchRequestFilters`, `PendingAmbiguity`, `AmbiguityCandidate`, `ResolveAmbiguityPayload`, `SSEEvent`, `SSEEventStep`, `SSEEventStatus`, `ProgressMessage`), value exports (`RETRY_MAX`, `formatApiError`), and new hooks (`useUpdateFetchRequest`, `useFetchRequestAmbiguities`, `useResolveAmbiguity`)
### `src/features/fetch-requests/useFetchRequests.ts` **(+49 lines)**
Added hooks:
- `useUpdateFetchRequest()` → `usePatch("fetch-requests")`
- `useFetchRequestAmbiguities(fetchRequestId)` → `useQuery` for `GET /fetch-requests/{id}/ambiguities`
- `useResolveAmbiguity()` → `useMutation` for `POST /ambiguities/{id}/resolve` with cache invalidation of both ambiguities and detail queries
### `src/main.jsx` **(+2 lines)**
- Added import for `FetchRequestDetail`
- Added route `{ path: "/fetch-requests/:id", component: FetchRequestDetail, headerTitle: "Fetch Request" }`
Reviewed-on: #9
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
|
|||
| d4a79c785d |
report-fetch-request-ui (#7)
## MR: Fetch Request Pipeline, Report Snapshots, and Admin Filtering
### Summary
Adds fetch request pipeline UI, report snapshot manager, snapshot selector on dashboard, and client-side in-memory filtering for the admin panel. Also overhauls the Home page with feature cards and adds navigation links.
### Changes
**New Pages**
- `/fetch-requests` — Upload bank statements (two-step: upload file, then configure source) or configure email ingestion. Table shows fingerprint (with copy), source type, account, status (color-coded chip), and created date.
- `/reports` — Generate cached report snapshots with filters (ignore self, date range, amount range). Table shows snapshot ID (with copy), creation time, and query summary chips.
**Dashboard**
- Snapshot selector autocomplete dropdown (formatted "Snapshot from {date}"), passes `snapshot_id` to `useReport`
- Styled to match other filter controls (caption above, auto-height)
**Admin — In-Memory Filtering**
- `FilterBar` component: collapsible, Dashboard-style column layout with caption + autocomplete/range/date inputs per filterable field
- `FilterAutocomplete` component: multi-select, free solo, checkmark ticks, selected-first sort frozen while dropdown open (prevents scroll reset)
- `applyClientFilters` in `ResourceView`: handles number range, datetime range, array (object/string elements), non-relation objects, boolean, primitive exact match
- Config-driven via `filterOptions: { mode: "client", fields: [...] }` in `openapi-config.ts`
- Mobile view: each filter takes full width (`flex: "0 0 100%"`), no horizontal squeeze
- `rowCount` omitted in client pagination mode (suppresses MUI X warning)
**Navigation & Home**
- Header nav links: Dashboard, Fetch, Reports
- Home page redesign: gradient hero, "Import Data" CTA, 4 feature cards (Dashboard, Fetch Requests, Report Snapshots, Admin) with accent-colored hover effects
**React-OpenAPI Library**
- `filterOptions` (mode + fields) on `ResourceOverride` and `ResourceConfig` types
- `EnhancedTable` mobile pagination (10 per page with Prev/Next, prevents browser hang with 10000 records)
- `useResource` accepts `filterOptions` from loader
**Misc**
- `public/favicon.png` added, proper `image/png` type in index.html
- 24 files changed, ~1541 insertions, ~100 deletions
### Files Changed (24)
| File | Change |
|------|--------|
| `src/FetchRequests.tsx` | +336 — new page |
| `src/ReportSnapshots.tsx` | +273 — new page |
| `src/features/fetch-requests/` | +96 — models, hooks, index |
| `src/features/report-snapshots/` | +40 — models, hooks, index |
| `src/Dashboard.tsx` | +58 — snapshot selector |
| `src/Home.tsx` | +224 — redesign with feature cards |
| `src/Header.tsx` | +26 — nav links |
| `src/main.jsx` | +4 — routes |
| `react-openapi/components/FilterBar.tsx` | +313 — new component |
| `react-openapi/components/ResourceView.tsx` | +151 — client filtering |
| `react-openapi/components/EnhancedTable.tsx` | +62 — mobile pagination |
| `react-openapi/types/config.ts` | +7 — filterOptions type |
| `react-openapi/types/overrides.ts` | +5 — filterOptions type |
| `react-openapi/utils/openapi_loader.ts` | +8 — load filterOptions |
| `react-openapi/hooks/useResource.ts` | +6 — filterOptions passthrough |
| `react-openapi/index.ts` | +3 — exports |
| `src/openapi-config.ts` | +15 — expenses config |
| `src/features/report/useReport.ts` | +13 — snapshot_id support |
| `index.html` | +1 — favicon link |
| `public/favicon.png` | +2910 bytes |
Reviewed-on: #7
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
|
|||
| 8bea3d06f6 |
Dashboard Refactor: Flow-based Metrics + Unified Data Model (#4)
# Dashboard Refactor: Flow-based Metrics + Unified Data Model
## Summary
This MR transforms the dashboard into a **flow-driven, backend-powered analytics system** with a significantly cleaner architecture and improved UX.
## Overview
This MR introduces a **major refactor of the dashboard and report data model**, transitioning from separate `expense/income` handling to a unified **flow-based (`outflows` / `inflows`) system** backed by a single `metric` structure.
It simplifies data handling, improves UI consistency, and enables better extensibility for future analytics.
---
## Key Changes
### 1. Data Model Simplification
* Replaced:
* `expenses` / `incomes`
* With:
* `metric`
```ts
ReportPeriod {
start: string;
end: string;
metric: {
sum: number;
count: number;
transactions: Transaction[];
}
}
```
* Eliminates duplication across logic paths
* Flow is now controlled at query level instead of data shape
---
### 2. Flow-based System (Core Change)
* Introduced:
```ts
type DashboardFlow = "outflows" | "inflows";
```
* Replaced all references of:
* `expense` → `outflows`
* `income` → `inflows`
* Flow is now:
* Controlled at **Dashboard level**
* Propagated to **API query (`useReport`)**
---
### 3. API Changes
#### `useReport`
* Removed legacy params:
* `group_by`, `rolling`, `include_transactions`, etc.
* New structure:
```ts
useReport({
periods: ["daily", "weekly", "monthly", "all"],
flow,
payee,
tags
})
```
* Backend now handles:
* Flow filtering
* Aggregation
---
### 4. Period System Update
* Removed:
* yearly, fyly, full
* Added:
* `daily`
* `all`
```ts
type PeriodType = "daily" | "weekly" | "monthly" | "all";
```
* Updated helpers:
* `periodIdToKey`
* `buildPeriodId`
* `buildLabel`
---
### 5. React Query UX Improvement
* Added:
```ts
placeholderData: keepPreviousData
```
* Prevents UI flicker on filter/flow changes
* Enables smooth transitions
---
### 6. Dashboard State Refactor
#### Before
```ts
mode: "expense" | "income"
```
#### After
```ts
flow: "outflows" | "inflows"
```
* Introduced `onFlowChange` callback
* Lifted flow state to parent (`Dashboard.tsx`)
* Flow change triggers API refetch
---
### 7. UI Improvements
#### Flow Toggle
* Replaced mode toggle with:
* Outflows / Inflows switch
#### Loading State Handling
* Added `isFetching` across components
* UI behavior during fetch:
* Reduced opacity
* Disabled interactions
#### Drill-down UX
* Added:
* "Clear Drill-down" button
---
### 8. New Components
#### TopPayees
* New analytics card
* Shows top payees based on:
* Selected period
* Drill-down filters
* Supports:
* Click-to-filter (drill-down)
---
### 9. Adapter Layer Simplification
#### Removed mode branching everywhere
Examples:
* `getAmount(period)` now uses:
```ts
period.metric.sum
```
* `LatestItems`, `TopTags`, `HistoryChart`:
* No longer split logic by expense/income
* Work on unified transaction stream
---
### 10. GroupKey Generalization
#### Before
```ts
{
payee?: string[];
tags?: string[];
}
```
#### After
```ts
{
[dimension: string]: string[];
}
```
* Enables future dimensions without refactor
---
## Behavioral Changes
* Flow selection now **controls backend query**
* All components consume **filtered data only**
* No client-side filtering for expense/income
---
## Benefits
* Single source of truth (`metric`)
* Cleaner adapters (no branching explosion)
* Easier feature additions (new dimensions, filters)
* Better UX (no flicker, smoother transitions)
* Backend-driven correctness
---
## Migration Notes
* Replace all `mode` usages with `flow`
* Update adapters to use `metric`
* Remove assumptions about:
* `expenses`
* `incomes`
* Ensure API supports:
* `flow`
* new period types
---
## Future Scope
* Add more dimensions (account, category hierarchy)
* Multi-flow comparison (inflows vs outflows together)
* Snapshot-based caching (already partially supported)
---
## Testing Notes
Verify:
* Flow toggle updates API calls
* No UI flicker on filter change
* Drill-down works across:
* tags
* payees
* Daily / Weekly / Monthly / All tabs behave correctly
* Loading state disables interaction properly
---
Reviewed-on: #4
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
|
|||
| 009ab50b47 |
major refactor of the dashboard and react-openapi integration (#1)
## Summary This MR introduces a major refactor of the dashboard and react-openapi integration, focusing on configurability, separation of concerns, and improved extensibility. --- ## Key Changes ### 1. OpenAPI / Admin Refactor * Extracted `ConfigContext` into a dedicated provider. * Introduced `AppProvider` to encapsulate: * Config loading * API client initialization * React Query setup * Removed internal `QueryClientProvider` from `Admin` for better composability. * `Admin` now supports both: * Standalone usage * Nested usage inside an existing provider ### 2. Resource System Improvements * Added `hidden` flag to `ResourceConfig` and overrides. * Admin UI now filters out hidden resources. * Added `useResourceByName` helper for dynamic resource access. * Improved `useResource`: * Handles undefined config safely * Adds guards for missing endpoints * Disables queries when endpoint is absent ### 3. API Client Enhancements * Added custom `paramsSerializer`: * Serializes arrays without `[]` * Ensures backend-compatible query formats ### 4. Dashboard Architecture Overhaul * Replaced hardcoded dashboard with **config-driven system**: * Introduced `ConfigurableDashboard` * Dashboard sections defined via config * New state model: * `mode` (expense/income) * `periodType` (rolling/calendar) * `comparison` * `selectedPeriodId` ### 5. Component Refactor (View / Logic Split) * Split major components into: * `.tsx` (logic/controller) * `.view.tsx` (presentation) * `.models.ts` (types) * Applied to: * Dashboard * HistoryChart * ProgressCard * LatestItems ### 6. HistoryChart Redesign * Fully rebuilt using report-driven data * Supports: * Weekly / Monthly / Yearly / FY / Full views * Rolling vs Calendar periods * Comparison mode (auto-aligned offsets) * Introduced: * Bucket merging logic * Dynamic comparison attachment ### 7. Reporting Integration * Dashboard now powered by: * `useReport` * `prepareReport` * Removes need for multiple manual API calls ### 8. UI / UX Improvements * Theme-aware color system * Dynamic accent colors per mode * Cleaner layout using section-based rendering * Improved selection and interaction in charts ### 9. Cleanup & Removals * Removed legacy components: * Old `HistoryChart` * Old `ProgressCard` * Simplified Header layout spacing --- ## Behavior Changes * Hidden resources are no longer visible in Admin UI. * Dashboard is now entirely configuration-driven. * API query params for arrays no longer use `[]`. * Resource hooks no longer crash on missing config. --- ## Risks / Considerations * Dashboard depends on correct configuration structure. * Hidden flag may unintentionally hide resources if misconfigured. * Query param serialization change must align with backend expectations. --- ## Follow-ups * Add typing improvements to remove `@ts-ignore` in `GenericForm`. * Extend dashboard config with more reusable section presets. * Add tests for report aggregation and comparison logic. --- Reviewed-on: #1 Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> |
|||
| 8a285bbdbe | init |
