PERF-WSF-002 — Page and authorize before materializing lists
| Field | Value |
|---|---|
| Status | Open; source-verified, scale not measured |
| Severity | High at scale |
| Components | sparqd/workspace-file-management-backend, sparqd/bff-api, sparqd/oqullus, PostgreSQL |
| Associated issues | Not opened |
Concrete observed case
Folder and Shared endpoints return full result sets, then Oqullus sorts, filters, and pages in memory. Search loads every name match, filters ACLs in Python, builds every path, then slices the requested page. Trash applies one offset/limit independently to folders and files and returns no combined total; Oqullus requests only the default page and paginates that subset locally. History similarly requests only the first 100 rows.
Expected condition
Authorization, deterministic ordering, filtering, and pagination execute before the result set is materialized. Responses include one stable cursor or total contract appropriate to concurrent changes.
Why it matters
The current APIs work for small workspaces but make request, database, BFF, and browser cost grow with the complete collection. Trash and history can also hide older records while presenting client pagination as complete.
Existing controls and workarounds
Keep folders small, navigate directly by stable ID, and treat Trash and history as potentially incomplete beyond their first fetched pages.
Alternatives considered
- Recommended: database-level ACL-aware keyset pagination and bounded path projection.
- Offset pagination with a combined total — easier but unstable and slower at deep offsets.
- Client-only pagination — acceptable only with an enforced small collection limit, which does not exist.
Component involvement
- The backend implements ACL-aware database queries, ordering, cursors, and totals where appropriate.
- BFF API validates and preserves the pagination contract.
- Oqullus requests pages and renders honest counts and continuation state.
Reproduction and verification
Follow WSF-S010.
Resolution conditions
Close when contract tests and load tests prove complete, stable pages and bounded resource use for folder, Shared, Trash, search, and history.
Validation and blockers
Query and client behavior are source-confirmed. Dataset thresholds and latency objectives are not yet defined or measured.