PERF-WSF-001 — Bound file-transfer memory and concurrency
| Field | Value |
|---|---|
| Status | Open; source-verified, load boundary not measured |
| Severity | High |
| Components | sparqd/oqullus, sparqd/bff-api, sparqd/workspace-file-management-backend, S3/OSS |
| Associated issues | bff-api #230, workspace-file-management-backend #19, oqullus #524 cover notebook-save scale and overlap this finding |
Concrete observed case
Oqullus uploads all selected files concurrently. BFF multer.memoryStorage
holds each complete upload before forwarding it. Downloads stream through the
backend and BFF, then Oqullus materializes the complete response as a Blob.
There is no explicit file-size, batch-size, or concurrency limit in the reviewed
path.
The backend already has multipart service primitives and the BFF exposes a presigned download resolver, but neither scalable path is wired through the UI.
Expected condition
Transfer limits and concurrency are explicit. Large bytes move directly between the browser and object storage under short-lived scoped capabilities, with progress, cancellation, integrity checks, and committed metadata finalization.
Why it matters
Concurrent batch uploads multiply BFF memory by file count and size. Proxied downloads consume application bandwidth and browser memory even when object storage can serve the authorized bytes directly.
Existing controls and workarounds
Use small files, upload one at a time, and avoid parallel large downloads. A presigned download endpoint and backend multipart primitives exist but are not connected end to end.
Alternatives considered
- Recommended: proxy small files under a strict limit; use direct multipart upload and presigned download for larger files.
- Stream every transfer through the BFF — simpler policy point but higher cost and weaker isolation under concurrency.
- Raise BFF memory — delays rather than removes the multiplicative risk.
Component involvement
- Oqullus selects transfer mode, bounds concurrency, and presents progress.
- BFF API enforces small-proxy limits and brokers scoped capabilities.
- The backend finalizes metadata and cleans abandoned multipart uploads.
- Platform storage policy controls expiry, CORS, and multipart lifecycle.
Reproduction and verification
Follow WSF-S003.
Resolution conditions
Close when measured limits, bounded memory, cancellation, expiry, partial-upload cleanup, and integrity behavior are proven under concurrent load.
Validation and blockers
The buffering path is source-confirmed. Safe load limits have not been measured.