Workflow Resource Monitoring
Resource Monitoring is a Workflow subfeature that presents per-task CPU,
memory, and disk allocation, usage, wastage, and provisioning status. It is an
observation and capacity-planning surface; it must never invent production
measurements or grant broader access than the underlying workflow.
Assessment scope
| Field | Value |
|---|
| Actors | Workflow viewer, editor, platform operator, security reviewer, and monitoring services |
| Parent capability | Workflow |
| Assessed environment | Realm gj76anadpsc; workspace w-t5e2kcxjd |
| Source baseline | Oqullus 8c9f240410b2; BFF 9b2fd45a4da1; workflow-resource-monitoring 8665dd6efe13; platform-stacks a71390d12463 |
| Last verified | 3 September 2026 |
This assessment covers metric collection, classification, storage, current and
historical reads, browser presentation, authorization, deployment, failure
states, recovery, scale, retention, and ownership. It does not assess the Spark
Recommendation text-generation or recommendation-approval journey.
Current capability boundary
| Layer | Source-backed state | Current KCL/deployment evidence |
|---|
| Oqullus | Resources action and modal are implemented | Failure can display generated values in the production surface |
| BFF | Latest/history proxy routes are implemented | RESOURCE_MONITORING_URL is not supplied by current KCL |
| Dedicated backend | Collector, Delta writer, latest API, and history API exist | Current KCL does not deploy the API or collector |
| Current producer | spark-recommendation job receives the Workflow resource table | KCL-defined; live table-write evidence pending |
| Current reader | Dedicated API source expects Trino and the monitoring table | Implemented but inactive in current KCL |
| Authorization | BFF validates a JWT | No managed-workflow resolution or backend caller authentication |
| History | API returns complete matching history; UI windows it | Server-side range, pagination, and size limits absent |
The feature is therefore implemented in pieces but not integrated as a
supported deployment path. A visible sample-data warning is not evidence that
the backend is absent; it is evidence that Oqullus entered its fallback path.
User-visible states
| State | Required presentation | Current behavior |
|---|
| Loading | Progress without stale conclusions | Implemented |
| Real data | Snapshot age, task, metrics, and provenance | Implemented in UI; deployed backend evidence pending |
| No data | Explain that the workflow has not produced observations | Structured API 404 is treated as no data |
| Stale data | Show last successful observation and age | No complete stale-data contract |
| Unavailable | Dependency and retry guidance; no measurements | Non-404 failure enters generated sample fallback |
| Demo | Persistent environment-level indication | Browser fallback and backend sample mode are separate and can be confused |
Findings
| ID | Finding | Components | Evidence | Impact |
|---|
ARCH-WF-002 | Wire one canonical Resource Monitoring backend through KCL | Oqullus, BFF, monitoring, recommendation, platform-stacks | Backend source exists; current KCL wiring absent | High |
DATA-WF-002 | Bind Resource Monitoring to task executions, not workflows | Monitoring, recommendation, workflow-backend, BFF, Oqullus, Airflow, Spark | Source-verified | High |
DATA-WF-003 | Replace periodic snapshots with execution-lifecycle metrics | Monitoring, recommendation, platform-stacks, Prometheus, Kubernetes, Spark | Source-verified; collector not KCL-deployed | High |
PROD-WF-003 | Never substitute generated metrics for unavailable monitoring | Oqullus, BFF, monitoring, platform-stacks | Source-verified | High |
SEC-WF-003 | Protect monitoring now and add resource scoping before multi-project rollout | BFF, workflow-backend, monitoring, platform-stacks, OpenFGA | Source-verified; deployed role evidence pending | High |
Interim guidance
- Do not use values carrying Showing sample data or sample trend for
capacity, reliability, or cost decisions.
- Treat Resources as unavailable until a captured request proves that the BFF
reached the authoritative backend and returned a real observation timestamp.
- Do not activate the dedicated API without internal authentication and a
managed-workflow authorization boundary.
- Do not deploy both monitoring collectors against the same table until one
producer, schema, merge key, and retention owner are explicitly selected.
- Treat existing rows as coarse daily task snapshots, not measurements of a
specific workflow run or task attempt.
- Treat each collected value as a point-in-time sample with mixed source
windows, not a complete execution profile.
- Query the authoritative monitoring table only through an approved service
identity when operational evidence is required before product integration.