ARCH-SQL-003 — Define a managed Trino compute lifecycle
| Field | Value |
|---|---|
| Status | Improvement backlog; source-verified architecture gap |
| Severity | High |
| Affected surface | Compute isolation, routing, lifecycle, permissions, quotas, recovery, and SQL Editor selection |
| Implementation issue | Not opened |
| Components | sparqd/platform-stacks; sparqd/oqullus; sparqd/bff-api; Superset; Trino Gateway; Trino; Ranger; catalog and metastore services |
Where the problem appears
The current SQL Editor selects Catalog, Schema, and execution Mode. It has no logical Compute selector or lifecycle. Platform deployment configuration can create and register a named additional Trino cluster, but that is an operator primitive rather than a workspace product capability.
Concrete observed case
The reviewed platform flavor configures one Trino cluster named
defaultcluster. The deployed realm gj76anadpsc, workspace w-t5e2kcxjd,
also exposed one query cluster behind the gateway. No user-facing create,
resize, start, stop, status, route, recover, or delete lifecycle was identified.
Expected condition
Keep one shared data-access plane and make compute isolation a separate concern:
- Trino clusters may share the same authorized catalogs and Ranger policies.
- A logical compute resource has an identity, workload class, size policy, lifecycle state, permissions, quota, health, and routing registration.
- SQL Editor can select a logical compute only when more than one supported choice exists; Catalog remains a Trino catalog and never becomes a compute or Superset database identifier.
- Users choose product intent such as Interactive, Batch, or an approved named compute. Raw worker counts, pod resources, and JVM configuration remain platform-managed.
Cause and evidence
- Platform stacks contains a named-cluster deployment and gateway-registration primitive.
- The default platform flavor wires one cluster, and the current product path has no lifecycle or routing contract for multiple computes.
- Gateway routing currently establishes reachability; it does not establish workspace permissions, quotas, readiness, recovery, or user selection.
Why it matters
Additional clusters can provide team, workload, or failure isolation, but raw deployment alone creates ambiguous ownership and routing. Conflating compute with Catalog would also fragment or duplicate the data-access model even though Ranger remains user-scoped and catalogs can be shared across clusters.
Existing controls and workarounds
An operator can deploy a named cluster with a selected size and fixed worker count and register it with the gateway. This is suitable for controlled platform administration, not self-service lifecycle management or an SQL Editor support promise.
Alternatives considered
- Recommended: retain one default compute until isolation is required, then add a platform-managed logical compute lifecycle and explicit routing.
- Expose raw cluster creation and Kubernetes sizing in SQL Editor. Rejected as the wrong abstraction and an unsafe operational boundary.
- Reuse Catalog as the compute selector. Rejected because Catalog identifies data, while clusters identify execution capacity.
- Require every cluster to own separate catalogs. Rejected for the current model: clusters can share catalog configuration while Ranger still enforces effective-user access.
Component involvement
- Platform stacks currently provides named deployment, fixed sizing, and gateway registration. It would need guarded scaling, health, and deletion mechanics for a managed lifecycle.
- A platform control-plane contract is needed for lifecycle state, policy, quota, permissions, reconciliation, and recovery.
- Trino Gateway currently routes to registered computes; readiness-aware lifecycle routing remains part of the proposed contract.
- Oqullus, BFF, and Superset would need to carry the selected logical compute and present its state.
- Catalog, metastore, and Ranger remain the shared data-access plane.
Reproduction and verification
Follow SQLC-S050. Current-state verification checks the deployed cluster inventory, gateway registrations, and product UI. Future acceptance must exercise the complete lifecycle and prove that changing compute does not change the caller's catalog authorization.
Resolution conditions
Close when product either documents one platform-managed default compute as the intentional boundary, or supplies a tested logical compute lifecycle covering create, ready, route, resize, stop, start, degraded, recover, and delete states, with permissions, quotas, audit, rollback, and shared Ranger/catalog semantics.
Validation and blockers
No multi-compute product contract, lifecycle API, workspace permission model, quota model, or recovery evidence was supplied. Product, platform, security, data, and operations validation is pending.