DATA-ML-001 — Connect the ML lifecycle identity
| Field | Value |
|---|---|
| Status | Open finding; source-verified; completed live run pending |
| Severity | High |
| Components | sparqd/platform-stacks (JupyterHub), sparqd/agent-based-ml-platform, MLflow, sparqd/bff-api, sparqd/oqullus |
| Associated issues | Not opened pending finding review |
Concrete observed case
AutoML creates a short workflow ID and separate nested MLflow runs. Its run tags use user email, while BFF discovers non-admin experiments and registered models through owner tags on those parent resources. AutoML does not set the matching experiment owner tag.
The backend also holds the active user email in process-global state while a pod can execute overlapping workflows. A later request can therefore change the identity read by nested run logging from an earlier request. The eight-character workflow ID also provides a small collision domain for a durable lifecycle key.
Supervised training logs an artifact without registering a model. The unsupervised path registers one fixed model name without owner metadata. Oqullus does not link an AutoML result to its MLflow parent/training run or a registered model version.
Notebook kernels receive general Cogrion and Jupyter OAuth context, but the deployment source does not configure an MLflow tracking URI, an MLflow-audience credential, or a supported notebook helper. A user can only attempt direct MLflow logging or registration after manual client setup.
The Models UI and BFF expose registered-model listing and detail reads. They do
not provide a Register model action, explain that logging an artifact does
not register it, or link a notebook run to its destination. A model registered
directly from a notebook can also omit the authoritative owner and workspace
metadata used by the BFF's non-admin listing filter, making a successful MLflow
write absent from the user's Models page.
Expected condition
Persist one authoritative mapping among workspace, user, notebook or AutoML workflow, attempt, MLflow experiment, parent/training run, logged artifact, registered model, and version. Expose View experiment, Register model, and View model version transitions with consistent ownership for every training path. Identity must be run-scoped and use a full globally unique workflow/attempt key.
Why it matters
Users cannot reliably audit how an AutoML result was produced, find their run,
register the intended artifact, or distinguish two users' candidates. A shared
model name also creates collision and governance ambiguity. Notebook users can
reasonably assume that mlflow.*.log_model() creates an item in Models, then
have no UI guidance explaining why nothing appears or how to promote the
artifact safely.
Existing controls and workarounds
Nested MLflow runs preserve some internal lineage, and model versions can refer
to a source run. An expert can manually configure the MLflow client and call
mlflow.register_model(), and administrators may locate records manually.
Neither workaround provides the platform's authoritative owner/workspace
metadata or a supported user journey.
Alternatives considered
- Provide a Cogrion registration API plus UI and notebook SDK/helper. The server validates the source run and stamps authoritative workspace and owner identity; this is the clearest governed path.
- Bootstrap notebooks for direct MLflow access with the correct tracking URI and audience-scoped credential. This is convenient for MLflow-native users, but authorization and required metadata must be enforced server-side rather than trusting notebook-supplied tags.
- Keep artifacts unregistered until an administrator promotes them. This is simpler operationally, but Models must not be presented as the user's next stage and the UI must explain the hand-off.
Component involvement
- Platform stacks provide a supported notebook tracking/authentication contract.
- AutoML emits stable lineage and consistent ownership metadata.
- MLflow stores experiment/run/model/version relationships and permissions.
- BFF authorizes registration and stamps authoritative owner/workspace identity.
- Oqullus explains logged artifact versus registered model and provides visible registration, navigation, and lifecycle status.
Reproduction and verification
Follow ML-S004.
Resolution conditions
Close when notebook, supervised, and unsupervised runs are visible to their author; the UI provides a documented registration path; all transitions use stable scoped identities; concurrent runs cannot exchange attribution; and model versions navigate back to the exact training attempt.
Validation and blockers
The identity discontinuities and missing notebook bootstrap/UI path are source-confirmed. Product must decide who may register a candidate, when it becomes a registered model, and its workspace naming scope.