PROD-SR-001 — Map advice to future Workflow task configuration

Field	Value
Status	Open; source-verified
Severity	High
Components	sparqd/oqullus, sparqd/workflow-backend, sparqd/spark-recommendation
Associated issue	Not opened

Where the problem appears

Open a Workflow recommendation, select New Config, and copy the generated Python. Oqullus converts every parsed setting into spark.conf.set(key, value).

Current behavior and evidence

Driver/executor cores and memory, memory overhead, node type, disk, and executor scaling belong to the saved Spark Workflow task and generated SparkApplication. They must be selected before the next Spark application starts. Changing them through a running SparkSession is ineffective or unsupported. Spark's configuration guidance places application properties in SparkConf/submission configuration rather than treating every property as a mutable SQL runtime setting.

Workflow-backend already models those fields and validates sizes, scaling combinations, node types, images, and a spark_conf map. Oqullus exposes the resource fields but does not serialize spark_conf for Spark tasks. The modal nevertheless labels generated runtime Python as the way to “Apply configurations.” Generic C/M/R/I node-family advice is also not mapped to the environment-approved node types.

The same modal is surfaced from Workspace, but Workspace initializes recommendationData to null; that entry point cannot currently show advice.

Expected condition

Advice is structured and maps each suggestion to one supported destination:

• a saved Workflow Spark-task field applied before the next run;
• a validated spark_conf override applied at submission;
• a runtime-mutable SQL setting, with exact timing; or
• informational-only advice with no apply claim.

The advisory UI shows current and suggested values, supported field, evidence, limitations, and Applies to future runs. It does not expose an automatic apply action. Remove or disable the incomplete Workspace entry point until it has a valid task-execution context.

Alternatives considered

1. Recommended: field-aware advisory output with copyable values or navigation to the relevant Workflow task editor.
2. Create a reviewed Workflow draft automatically. Deferred because this assessment is advisory-only.
3. Keep generic Python. Rejected because submission-time resource settings do not become effective through that path.

Reproduction and verification

Follow SR-S001.

Resolution conditions

Close when every displayed setting has a validated Cogrion application point and phase, unsupported advice is clearly identified, submission settings are not represented as runtime Python, and the incomplete Workspace surface is removed or supplied with canonical execution context.