REL-CAT-002 — Retry only transient, safe Catalog failures
| Field | Value |
|---|---|
| Status | Open finding; source-verified with repeated live failures |
| Severity | High |
| Affected surface | All Catalog requests using the shared HTTP wrapper |
| Implementation issue | oqullus #534 |
| Code-change repository | sparqd/oqullus |
Where the problem appears
Trigger a deterministic failing Catalog request and count identical attempts by URL and payload.
Concrete observed case
The shared client retries every HTTP 500 twice. One logical operation therefore creates three requests. Two eager failing DQ reads can create six failed executions before other table-detail failures are counted.
Expected condition
Retry only classified transient failures for idempotent operations, honor server backoff, cap attempts, and expose retry state. Validation, authorization, unsupported-operation, and deterministic engine failures return once.
Cause and evidence
- The retry rule is based on HTTP 500 rather than failure class.
- Observed identical failed requests matched the configured original plus two retries.
- Catalog includes query-producing operations where retries can amplify compute work.
Why it matters
Retries multiply load, latency, logs, query history, and incident noise, and can repeat an expensive engine failure such as resource exhaustion.
Existing controls and workarounds
Avoid repeated navigation while deterministic failures persist. Operators should group identical attempts by correlation identity when assessing impact.
Alternatives considered
- Recommended: typed server errors and operation-aware retry policy.
- Disable all retries — safer for execution but unnecessarily weak for transient metadata reads.
- Retry at the BFF only — viable if it has authoritative failure classification and end-to-end attempt telemetry.
Component involvement
- BFF defines stable error categories and retryability.
- Oqullus applies idempotent operation-aware policy and visible state.
- Operations tracks logical operations separately from attempts.
Reproduction and verification
Follow CAT-S020. Record the visible state, request outcome, effective identity, and recovery behavior relevant to this finding.
Resolution conditions
Close when deterministic failures produce one upstream attempt, transient retries are bounded and observable, and query-producing operations cannot be repeated without an explicit safe contract.
Validation and blockers
Oqullus is primary; BFF and operations owners validate classification and telemetry.