AI/ML
The current AI/ML area contains three related machine-learning lifecycle surfaces: AutoML trains a candidate model, MLflow Experiments records runs and artifacts, and MLflow Models displays registered model versions. They are related by lineage, but the product does not yet provide a complete transition from an AutoML workflow to a visible experiment, registered model, and served endpoint.
AI Gateway is separate. AutoML calls it for GenAI planning and reasoning; AI Gateway routes external chat-model requests and does not deploy AutoML or MLflow-trained models.
Start with Use AI/ML. See Architecture, evidence and reproduction, and AI/ML components.
Assessment scope
| Field | Value |
|---|---|
| Actors | Data scientist, ML engineer, platform administrator, security reviewer, and platform operator |
| Components assessed | Oqullus, BFF API, AutoML backend, MLflow, Trino/Ranger, object storage, PostgreSQL, platform stacks, and the AutoML-to-AI-Gateway dependency |
| Assessed environment | Realm gj76anadpsc; workspace w-t5e2kcxjd; Kubernetes context dev-north-alicloud |
| Deployed baselines | AutoML 6b3fa2a; AI Gateway 0b9c3d2; MLflow 3.14.0 |
| Source baselines | Oqullus f76e57db; BFF API 93ea18e7; AutoML 3112ee9f; AI Gateway 94236c69; platform stacks 5f3f0f0e |
| Exclusions | Full AI Gateway provider, budget, guardrail, and routing review; production model serving; model-quality benchmarking |
| Last verified | 8 September 2026 |
| Review status | Engineering assessment recorded; product, ML, security, data-governance, and operations validation not recorded |
Current lifecycle
- A user selects a CSV or Trino table and starts an AutoML workflow in Oqullus. BFF forwards the request to the AutoML backend.
- The AutoML backend creates its own short workflow ID, persists workflow and agent state, and runs training in the API pod that accepted the request.
- The backend uses AI Gateway for LLM-assisted planning, a shared Trino service identity for table access, local Spark or Python for compute, object storage for data and generated artifacts, and MLflow for run tracking.
- Synthetic Data separately accepts a CSV, trains a CTGAN or TVAE synthesizer, and writes a shuffled augmentation containing original and generated rows.
- Oqullus polls AutoML status and stores the active workflow ID in that browser. Terminal history is retrieved separately by email.
- Experiments and Models read MLflow through BFF. AutoML's workflow identity, ownership metadata, experiment visibility, and registration behavior do not consistently match those pages.
- Models is currently a read-only registry browser. Oqullus does not expose a trained-model deployment or serving journey.
Capability boundary
| Capability | Current state | Boundary |
|---|---|---|
| CSV AutoML | Available | Upload and end-to-end size limits are not declared; execution is tied to one API pod |
| Trino AutoML | Available through a service identity | It does not preserve the signed-in user's Trino/Ranger identity |
| Task suggestions | Available for an uploaded CSV | Suggestions profile the upload; they do not reserve compute or prove the suggested workflow will complete |
| Data-engine selection | Automatic internal algorithm tiers | Users cannot select a compute profile; default Spark is local to the API pod, including tiers labelled large or massive |
| Synthetic Data | CSV generation, polling, quality summary, and history are visible | Output includes original rows; low or unavailable quality is warning-only; UI does not provide an authorized download action |
| Workflow status | Polling and terminal history are available | Active recovery is browser-local; direct backend reads are unauthenticated in the assessed deployment |
| Cancellation | Backend marks cancellation and stops between agents | No Oqullus control; the active step is not interrupted immediately |
| Retry and pod recovery | No durable queue or run lease | Pod loss can leave persisted work in a non-terminal state |
| Experiments | Read-only browsing and run details | AutoML ownership conventions do not match the page's filtering; result completeness is not signalled |
| Models | Read-only registered-model and version browsing | Supervised AutoML does not register a model; no lifecycle action is exposed |
| Serving | Not exposed | A disabled UI route and backend prototypes do not constitute a supported journey |
| AI Gateway | External GenAI routing | It is an AutoML dependency, not the destination for trained models |
| Compute capacity | Two fixed AutoML API replicas, each requesting 1 CPU and 8 GiB | No durable worker pool, admission control, queue depth, or workload-based scaling |
| MLflow capacity | One live application pod was observed | KCL declares 1–2 autoscaling, but no HPA was present during the live check |
| Persistence and recovery | PostgreSQL plus object storage | AutoML and MLflow databases each have one replica, no database-native backup policy, and destructive termination policy; no coordinated restore is evidenced |
Critical journeys
| Journey | Required result |
|---|---|
| Start from CSV or Trino | Validate input and authorization before accepting durable work; state size and compute limits |
| Observe and reload a run | Recover authoritative queued/running/terminal state across reload, browser, and pod changes |
| Cancel | Propagate cancellation to active work and report the confirmed terminal outcome |
| AutoML backend or pod failure | Reconcile, retry safely, or fail the run visibly without leaving it indefinitely running |
| Partial output | Mark each artifact complete, failed, or quarantined and never present an earlier/partial artifact as the terminal result |
| AI Gateway failure | Attribute the planning failure to the dependency and preserve a safe retry point |
| Trino/Ranger denial | Apply the intended user policy and return a non-leaking authorization error |
| Concurrent users | Preserve run-scoped user identity, enforce admission limits, and avoid capacity interference |
| Invalid data or configuration | Reject before durable execution with field-specific limits and no orphaned upload |
| Generate synthetic data | State whether output is synthetic-only or augmented, enforce quality/privacy policy, and provide an authorized download |
| Cancel synthetic generation | Stop fitting and output publication without later overwriting cancelled state as completed |
| Reproduce a result | Retain immutable data, configuration, code, environment, dependency, seed, and attempt lineage |
| Review an AutoML experiment | Navigate from workflow to the correct authorized MLflow parent and training run |
| Register a candidate | Create an owner/workspace-scoped model identity and version with complete run lineage |
| Browse many resources | Page and filter the complete authorized experiment, run, model, and version collections |
| Serve a model | Remain explicitly unavailable until deployment identity, runtime, rollout, monitoring, and rollback exist |
| Database or artifact loss | Restore consistent workflow, tracking, registry, and artifact state to a tested recovery point |
Findings
| ID | Finding | Components | Evidence | Impact | Issue |
|---|---|---|---|---|---|
SEC-ML-001 | Authenticate the AutoML ingress and authorize every run and artifact | AutoML backend, BFF API, platform stacks, Oqullus | Source + live unauthenticated GETs | Critical | Not opened |
SEC-ML-002 | Apply one MLflow resource-authorization policy across all BFF routes | BFF API, MLflow, Oqullus | Source-verified; cross-user live test pending | Critical | Not opened |
SEC-ML-003 | Preserve an explicit user-scoped training-data authorization policy | AutoML backend, BFF API, Trino, Ranger, platform stacks | Source + deployment configuration | Critical policy gap | Not opened |
DATA-ML-001 | Give notebook and AutoML artifacts a supported path into MLflow Models | JupyterHub, AutoML backend, MLflow, BFF API, Oqullus | Source-verified | High | Not opened |
DATA-ML-002 | Define synthetic-data output, quality, privacy, and retrieval semantics | AutoML backend, BFF API, Oqullus, object storage | Source-verified; live generation pending | High | Not opened |
REL-ML-001 | Make AutoML execution, cancellation, and recovery durable | AutoML backend, Oqullus, platform stacks, PostgreSQL | Source + deployed topology | Critical | Not opened |
REL-ML-002 | Preserve and explain failed AutoML runs | AutoML backend, BFF API, Oqullus | Live + source-verified | High | Not opened |
ARCH-ML-001 | Define the AutoML engine and compute boundary | AutoML backend, Spark, Trino, MLflow, platform stacks, Oqullus | Source + deployed configuration; option benchmarks pending | High; critical for large-data claims | Not opened |
PERF-ML-001 | Separate API and training capacity and expose an admission envelope | AutoML backend, platform stacks, Spark, Oqullus | Source + deployed resources | High and cost | Not opened |
PERF-ML-002 | Page and filter complete MLflow collections | Oqullus, BFF API, MLflow | Source-verified | High at scale | Not opened |
OPS-ML-001 | Define retention, backup, and recovery across ML state | Platform stacks, AutoML backend, MLflow, PostgreSQL, object storage | Source + deployed topology | Critical | Not opened |
UX-ML-001 | Present the actual ML lifecycle and keep AI Gateway distinct | Oqullus, platform documentation, AutoML backend, MLflow, AI Gateway | Source-verified | Medium | Not opened |
UX-ML-002 | Expand the EDA pipeline-stage acronym | Oqullus | Source-verified | Low | Not opened |
UX-ML-003 | Make target and clustering modes explicit | Oqullus, AutoML backend | Source-verified | Medium | Not opened |
UX-ML-004 | Use an AI/ML umbrella without conflating its capabilities | Oqullus, platform documentation | Source-verified | Low | Not opened |
Findings describe observed behavior and expected outcomes, not accepted implementation commitments. Issues remain unfiled until finding review.
Assessment maturity
Scale: 0 absent, 1 ad hoc, 2 partly documented or automated, 3 routinely verified, 4 measured and improving.
| Lens | Score | Evidence-backed assessment |
|---|---|---|
| Product and UX | 1 | AutoML and read-only MLflow browsing exist, but their transitions, active recovery, cancellation, registration, and serving boundary are incomplete. |
| Architecture and data | 1 | Lifecycle identity is fragmented; synthetic output semantics and local-versus-distributed engine boundaries are not governed coherently. |
| Security and governance | 0 | The deployed AutoML backend permits unauthenticated reads, training uses a shared Trino identity, and MLflow resource checks are inconsistent. |
| Reliability and operations | 1 | Fixed replicas are healthy, but work runs in API processes and backup, reconciliation, retention, and restoration are not established. |
| Performance and cost | 1 | Fixed high-memory API pods exist without queue/admission signals; discovery APIs truncate collections without complete pagination. |
| Delivery and lifecycle | 1 | Exact deployed revisions are known, but the runtime architecture differs from stale worker documentation and capability claims exceed the implemented product. |
Current evidence limitation
The AutoML backend claims were reverified against
sparqd/agent-based-ml-platform@3112ee9f7e721a5ad730073bca26bbb755218dd3;
see the commit-pinned backend evidence map.
That current-main baseline differs from deployed image 6b3fa2a; only
scenarios with live or deployment evidence establish deployed behavior.
Source and deployment evidence establish the current boundaries and the
unauthenticated-read condition. Controlled user-owned training, cross-user
authorization, pod-loss, cancellation, large-collection, quality, cost, and
restore exercises have not been completed. They remain Unverified only for
those specific outcomes; this label does not deny that the deployed features
exist.