Skip to main content

ARCH-WF-002 — Wire one canonical Resource Monitoring backend through KCL

FieldValue
StatusOpen; source-verified, deployed request evidence pending
Architecture stateBackend implemented but not wired by current KCL
SeverityHigh
Affected surfaceWorkflow list → Resources
Componentssparqd/oqullus, sparqd/bff-api, sparqd/workflow-resource-monitoring, sparqd/spark-recommendation, sparqd/platform-stacks
Associated issueNot opened

Where the problem appears​

From the Workflow list, open a row's action menu and select Resources. The modal requests the latest snapshot and history through the BFF. When that path cannot reach its backend, the UI displays:

Showing sample data — the resource-monitoring service is not connected yet.

Concrete observed case​

The backend is implemented in sparqd/workflow-resource-monitoring at source baseline 8665dd6efe13: it contains a scheduled collector, Delta writer, and FastAPI latest/history reader. The BFF also contains the corresponding proxy routes and expects RESOURCE_MONITORING_URL.

Current sparqd/platform-stacks KCL at a71390d12463 does not deploy the Workflow Resource Monitoring API and does not supply RESOURCE_MONITORING_URL to the BFF. Instead, it deploys the separate spark-recommendation API and job. The recommendation job is configured with the same cogrion_system.workflow_resource_monitoring table, but its API exposes recommendations rather than the monitoring latest/history contract.

The write-side table can therefore exist while the browser read path remains unconnected. A deployed network capture in gj76anadpsc is still required to prove the exact runtime failure; source alone does not prove whether an out-of-band deployment exists.

Expected condition​

KCL declares one complete Resource Monitoring path:

  1. one canonical producer owns the monitoring table and its retention policy;
  2. one API reads the table using the active Alibaba Trino authentication model;
  3. the BFF receives the API's internal service URL and authenticates the service call;
  4. readiness verifies Trino and table access, not only process health; and
  5. Oqullus exposes Resources only when a server-owned capability reports the path ready.

Cause and evidence​

  • sparqd/workflow-resource-monitoring implements the collector and FastAPI read endpoints.
  • sparqd/bff-api defines /resource-monitoring/:dagId and /history, while its client warns and fails at runtime when RESOURCE_MONITORING_URL is empty.
  • Current KCL contains no Workflow Resource Monitoring application or BFF environment value.
  • Current KCL does deploy spark-recommendation; its job receives WORKFLOW_RESOURCE_TABLE, while its API exposes only /recommendations.
  • A Helm chart and legacy Terraform wiring for the dedicated API exist, but those are not the current KCL deployment contract.

Why it matters​

Users receive generated sample values instead of monitoring whenever the BFF cannot reach the missing read service. Separate monitoring and recommendation implementations also create ambiguous table ownership, schema evolution, retention, incident response, and cost responsibility.

Existing controls and workarounds​

Treat the Resources modal as unavailable in gj76anadpsc until the live BFF request proves a real backend response. Do not use its fallback values for capacity or cost decisions. Operators may inspect the authoritative monitoring table directly through an approved service identity.

Alternatives considered​

  1. Recommended incremental path: deploy only the dedicated monitoring API in KCL, keep the currently configured recommendation job as the sole writer, add Alibaba Trino authentication, and wire the BFF URL. Do not deploy the second collector concurrently.
  2. Consolidate the latest/history endpoints into spark-recommendation, which already owns the deployed producer. This reduces deployments but couples monitoring availability and authorization to the recommendation product.
  3. Move both collection and reading to workflow-resource-monitoring, then remove monitoring-table production from spark-recommendation. This gives the cleanest ownership but requires a controlled producer migration.
  4. Replace the custom monitoring-only path with Prometheus recording rules and an authenticated Grafana dashboard embedded in Cogrion. This removes the periodic collector and duplicate charts, but per-workflow users still need a server-enforced authorization boundary. Use unmodified, separately deployed Grafana OSS subject to its AGPL terms; review required notices/source, plugins, modifications, branding, and any Enterprise/OEM features.
  5. Keep the browser fallback as the integration — rejected because generated metrics are not an operational backend.

Component involvement​

  • sparqd/platform-stacks declares the chosen API, service URL, credentials, probes, feature capability, and exactly one producer.
  • sparqd/workflow-resource-monitoring supplies the dedicated API and, only if selected, the canonical collector.
  • sparqd/spark-recommendation retains or relinquishes table production explicitly; it must not silently compete with another writer.
  • sparqd/bff-api authenticates and routes the internal call.
  • sparqd/oqullus shows unavailable/no-data/stale states and removes generated production fallback values.

Reproduction and verification​

Follow WF-S021.

Resolution conditions​

Close when KCL renders and deploys one canonical producer/API path, the BFF has a non-empty internal URL, live latest/history requests return authoritative rows, dependency failure shows Unavailable, and duplicate-writer checks pass.

Validation and blockers​

Frontend, BFF, monitoring, recommendation, data-platform, infrastructure, security, and operations reviewers must validate. Live deployment evidence in realm gj76anadpsc, workspace w-t5e2kcxjd, remains pending.