PERF-ML-002 — Page complete MLflow collections
| Field | Value |
|---|---|
| Status | Open finding; source-verified; scale test pending |
| Severity | High beyond current fetch limits |
| Components | sparqd/oqullus, sparqd/bff-api, MLflow |
| Associated issues | Not opened pending finding review |
Concrete observed case
Experiment listing fetches at most 1,000 without preserving a next-page token. Run expansion requests a bounded subset and then paginates it in the browser. Models fetches a first set of 25 and versions a first set of 100. Naming differences also cause some requested model sort/limit fields to be ignored. Search and filters operate on what was fetched, so an absent result can mean "outside this subset" rather than "does not exist."
Experiment filter options can also depend on metadata from runs that the user has already expanded, causing filter results to change with browsing history.
Expected condition
Preserve MLflow page tokens end to end with stable server-side authorization, search, sorting, and page sizes. Display complete counts only when known and label partial results explicitly. Filter options must not depend on which rows the user expanded.
Why it matters
Users can miss experiments, runs, models, or versions silently and make lifecycle decisions from an incomplete collection. Browser-side filtering does not scale.
Existing controls and workarounds
Current limits bound some response sizes. Administrators can use MLflow directly to locate omitted resources. The product UI does not disclose truncation.
Alternatives considered
- Recommended: cursor/page-token pagination and server-side search/filter/sort.
- Incremental infinite scrolling using the same stable cursor contract.
- Raise fixed limits — rejected because it postpones failure and increases latency and memory use.
Component involvement
- Oqullus owns page state, partial-result disclosure, and stable filters.
- BFF forwards tokens and validated search/sort parameters.
- MLflow supplies authorized cursor-based collection reads.
Reproduction and verification
Follow ML-S008.
Resolution conditions
Close when objects beyond every current limit are discoverable, stable under concurrent writes, and UI counts and filters never imply completeness falsely.
Validation and blockers
The token and subset behavior is source-confirmed. Representative large MLflow collections are not present or have not been tested in the assessed workspace.