Ontology architecture

System boundary

Component responsibilities

Component	Responsibility
Oqullus	Project and branch selection, object forms, structural client checks, list/search presentation, proposals, conflicts, semantic query, and workbooks
BFF API	Authenticate the workspace request, exchange an Ontology-audience token, and proxy explicit Ontology routes
Ontology backend	Authenticate the exchanged token, validate definitions, enforce branch ownership, persist projects/branches/objects/proposals, generate Cube YAML, issue Cube tokens, and proxy Cube APIs
PostgreSQL	Authoritative project, main projection, branch overlay, revision, proposal, notification, workbook, and dashboard state
Cube	Compile the effective dynamic model, plan semantic queries, cache query state, and create the Trino driver
Trino and Ranger	Execute physical SQL and enforce the effective user's data policy
Platform stacks	Deploy OAuth client, application secret, ontology-backend, PostgreSQL, Cube, storage, routing, and pinned images

Identity and authorization

The BFF validates the incoming workspace token and exchanges it for the ontology-backend OAuth audience. The backend maps actor identity from preferred_username, then sub, then email. Branch ownership compares this value directly with owner_user_id.

All platform personas are currently mapped to the ontology OAuth client. Authentication therefore controls service reachability, but no project membership table or project authorization check exists. Project routes and model reads accept the caller-selected project_id. This is recorded as an improvement because projects currently behave as shared organizational namespaces, not as claimed security partitions. Physical semantic queries still carry the caller identity into Trino, where Ranger provides data authorization.

Model state and authority

ontology_objects stores the published main projection. Each object has a stable UUID, project ID, display name, actor fields, timestamps, and one JSONB definition containing sql_table, dimensions, measures, and embedded relationships.

A working branch stores only its overlay in ontology_branch_object_changes. Effective branch state is reconstructed from the branch's recorded main base revision plus ordered create, update, rename, and delete changes. The project is the database isolation key for object names, branches, revisions, and proposals.

Relationships are single-source: the source object's JSON definition owns the target name, cardinality, and SQL. The backend checks that a target exists in the effective project/branch view and blocks deletion while incoming references remain. PostgreSQL cannot enforce those references with a foreign key because they are embedded in JSON.

Validation boundary

Request and service validation currently cover:

• allowed object and member names;
• allowed dimension, measure, and relationship types;
• duplicate member and relationship targets;
• at least one primary-key dimension;
• non-empty sql_table and required expressions;
• existing relationship targets; and
• balanced braces in template expressions.

The backend does not parse or execute expressions, resolve tables or columns, or compile the generated Cube model during save, proposal submission, or merge. Oqullus obtains database, schema, table, and column options through Superset, but a browser selector is advisory and can be bypassed or become stale.

Draft and publication validity should be separate states. Draft save may retain unresolved datasource work with visible diagnostics. Proposal submission and merge must validate current table/column metadata and compile the complete effective Cube model. Validation evidence must be tied to the exact branch revision and datasource identity.

Publication and versioning

Working-branch writes increment the branch revision. An approved merge creates one ontology_main_revisions row, applies all ordered branch changes to the published projection, creates immutable ontology_object_versions rows, rewrites saved references for renames, updates proposal and branch state, adds notifications, and commits once. This is the strongest transactional boundary in the reviewed lifecycle.

The older POST /v1/objects and DELETE /v1/objects/{id} routes write the main projection directly. Oqullus uses branch routes, but BFF continues to proxy the direct routes and backend integration tests use them to seed main state. Their object write and subsequent main snapshot use separate commits, so a snapshot failure can leave the projection ahead of revision history. Retire these routes after replacing bootstrap/test seeding, or restrict a retained import path and make projection plus revision atomic.

Proposal review boundary

A proposal is a live view of one working branch, not an immutable copied diff. Creating it records tasks from the current branch overlay. A later branch edit deletes its tasks and decisions, rebuilds the tasks, increments the branch revision, and returns the proposal to Open. Reviewers cannot decide on their own proposal; data stewards and tenant/platform administrators can approve or reject, while merge remains platform-administrator-only.

The database retains every decision row, but the read contract reduces them to the latest decision per reviewer. It omits decision time and proposal create, update, decision, and merge time. The UI therefore presents effective status, not a complete governance chronology.

For destructive changes, Oqullus calculates impact in the browser. It loads the first page of main objects, fans out one detail request per returned object, loads every workbook and every workbook detail, then searches relationships and saved query member names. Any read failure sets impact to null; merge remains available. This is useful preview logic but not an authoritative control because it can be incomplete, stale, or absent. The backend should compute impact for the exact proposal revision, return a completeness marker, and recheck it in the merge transaction. Product policy can then block dangling references or require an explicit audited override.

Proposal listing has no limit, cursor, or timestamp in its API representation. Proposal diff iterates branch changes and resolves base payloads per item. Both paths need bounded pagination before proposal history or large diffs can have a supportable scale target.

Workbook state boundary

Workbooks and tabs are mutable runtime state in Ontology PostgreSQL. A workbook contains project_id, workspace_id, owner_id, a name, and a JSON array of dashboard tab IDs. Each tab contains query configuration, chart choice, optional result and visualization fields, and timestamps.

The route layer does not authorize these resources. List accepts any caller- provided workspace/project filter; get, update, and delete use only a numeric workbook ID. Tab routes likewise check only that the tab belongs to that numeric workbook. BFF forwards the exchanged bearer token but no X-User-Id; the create dependency therefore falls back to local-dev-user. Oqullus ignores the stored owner and renders the current user as owner with Full access for every returned row. Authentication is present, but workspace, project, owner, and action policy are not enforced.

Current save behavior is mixed:

• selecting fields, filters, order, or limit changes browser state only;
• a successful Run asynchronously writes query configuration, but write failure is discarded;
• chart-type changes write immediately and discard failure;
• Reset changes browser state but does not clear the saved query;
• workbook and tab renames write on commit and report a toast on failure; and
• dashboard membership remains local until Save Dashboard.

There is no dirty marker, revision/ETag, autosave acknowledgement, conflict detection, or recovery draft. Reload restores the last successful query and automatically executes it, which can differ from the state visible immediately before reload.

dashboard_tab_ids is not a relational association. Update accepts arbitrary integers, deleting a tab does not remove its ID, and tab reorder ignores missing or foreign IDs while permitting duplicate positions. Concurrent clients use last-write-wins updates. Prefer a validated association plus uniqueness and optimistic concurrency, or enforce equivalent invariants transactionally while retaining JSON.

The current UI intentionally does not save result_data; reopening a configured tab reruns it. That avoids large stale result blobs in PostgreSQL, but means a workbook is a query definition, not a result snapshot. The unused result and visualization fields should either receive a bounded lifecycle contract or be removed from the public update schema.

Object graph boundary

The graph is a browser projection, not an independently stored graph. Oqullus requests the object-summary list, fans out one detail request per summary, creates edges by resolving each relationship's target name in the fetched subset, runs Dagre for both left-to-right and top-to-bottom candidates in Auto mode, performs label-collision passes, and renders all cards and edges with React Flow.

This inherits the first-50 object defect. A relationship to the 51st object is silently absent because its target is not in the browser map. A failed detail request is converted to a blank isolated card rather than an attributable partial-graph warning. The main thread also owns layout, searching, sorting, and rendering, so merely increasing the list limit exchanges a correctness problem for browser latency and memory pressure.

A graph-specific server projection should return stable node IDs, minimal card summaries, edges, total counts, revision, and truncation/completeness state in a bounded number of calls. Large graphs should open as a searchable neighborhood or progressively expanded subgraph. Full-project export can remain separate; the interactive canvas does not need to render the entire model at once.

Semantic-query execution boundary

Every query-builder change starts a debounced Cube dry run. Run has no default row limit and offers an explicit No Limit choice. Cube returns one result set, and Oqullus materializes table and chart pivots, sorts, totals, paginates, and exports over the full returned array. Ontology's proxy uses blocking requests inside the async service's thread pool, buffers the complete response, and has a 30-second timeout; BFF buffers the JSON again.

The visible Stop control races the Cube SDK promise against an AbortSignal but does not pass that signal to cubeApi.load. It only releases the UI waiter. Cube documents native AbortController support and a running-query cancellation API; neither is carried through this proxy. Trino is configured with a two-hour client timeout and a 24-hour maximum execution time. Whether an intermediate disconnect eventually cancels the Trino query is not correlated or proven.

Apply a server-owned default and hard maximum to interactive queries, use server/engine paging or bounded continuation for exploration, and route large exports through an asynchronous object-storage artifact. Carry a request/query identity across every hop and expose explicit cancellation whose completion is confirmed from Cube and Trino terminal state.

Cube model loading

Cube uses the supported dynamic-model extension points:

• contextToAppId separates compiler state by project and branch;
• repositoryFactory requests generated YAML files for the effective model;
• schemaVersion contains project/branch revision context and a model hash; and
• contextToOrchestratorId separates query state by user, project, branch, and a five-minute token-time bucket.

The proxy resolves a complete effective model before every load, dry-run, SQL, or metadata request even though token creation uses only branch metadata. Cube can then call repository and schema-version endpoints, each of which resolves the complete model again; schema-version serializes and hashes it. Compiler state is partitioned per project/branch, while query orchestration is partitioned per user/project/branch/five-minute bucket. Those boundaries protect model and user isolation but multiply database work, compiler copies, queues, caches, and driver contexts as users and active branches grow.

Split lightweight revision/security context from model materialization. Cache an immutable generated model snapshot by project, branch, and revision; reuse it for repository and schema-version; invalidate it on committed revision changes; and cap or expire active branch/compiler contexts. Keep user-sensitive result caches partitioned without forcing model generation to repeat per user.

Cube documents repositoryFactory for context-specific models and schemaVersion for recompiling changing dynamic definitions. See Cube custom data models and dynamic schema version.

The Cogrion configuration catches repository failures and returns []; it catches schema-version failures and returns a fixed default. That converts an upstream dependency failure into apparently valid empty/default compiler input. The user can receive a missing-cube error without being told that Ontology was unavailable. The runtime should either fail the request with an attributable dependency error or serve a deliberate last-known-good model marked stale and observable.

Listing and scale boundary

The branch list endpoint defaults to limit=50, offset=0, and reports the complete count. Oqullus sends neither pagination parameter nor server-side search and discards total. It then requests one object detail for each of the returned summaries to calculate counts, filters the resulting array locally, and renders eight objects per client page.

The backend also reconstructs and sorts the entire effective branch model in memory before slicing each requested page. Correct pagination therefore requires both layers to change: the UI must request and render server pages, and the backend needs a bounded effective-model query or maintained summary projection. Summary responses should include dimension, measure, relationship, and updated-time fields so the UI does not issue an N+1 detail fan-out.

Deployment and recovery

The assessed deployment has one ontology-backend pod with two Uvicorn workers, 250m CPU/256 MiB memory requests, and 500m CPU/512 MiB limits. Readiness and liveness probes check /. It has no HPA or disruption budget.

Cube runs one API pod, one CubeStore router, and two CubeStore workers. The API requests 1 CPU/2 GiB and is limited to 2 CPU/3 GiB. Each CubeStore component requests 500m CPU/1 GiB and is limited to 1 CPU/2 GiB. AliCloud's required pod affinity places the router and both workers on the same node; live placement confirmed that all four Cube/CubeStore pods were on one node. No Cube HPA or disruption budget was present.

The Cube refresh worker is disabled, scheduled refresh is disabled, generated Ontology models define no pre-aggregations, and cacheAndQueueDriver is explicitly memory. CubeStore is therefore not the distributed cache/queue for the semantic-query path, and its workers do not accelerate raw Trino queries. Scaling CubeStore workers alone would add cost without removing the current query bottleneck.

Trino has one coordinator and one fixed worker; live placement put both on one node. The KCL flavor exposes a preset and fixed worker count, but worker autoscaling is disabled. No resource-group admission policy, task retry policy, or exchange-manager wiring was identified. The exchange bucket exists as an infrastructure resource but is not used by the cluster configuration. The Trino Gateway has an HPA, but Cube connects directly to trino-defaultcluster.trino.svc.cluster.local, so gateway replicas and future multi-cluster routing do not scale semantic queries.

Cross-component scaling options

Component	Current scaling unit and constraint	Useful next option	Does not solve
Ontology backend	One pod, two workers; full model resolution and blocking proxy threads	Remove repeated model loads, pool async outbound connections, add model/revision cache, metrics, PDB, then HPA or fixed replicas	More replicas do not fix PostgreSQL/model amplification or a shared in-memory cache
Cube API	One pod; 1 CPU/2 GiB request; memory cache/queue; compiler per project/branch	Switch production cache/queue to CubeStore, bound compiler contexts, then run at least two APIs and scale from latency/queue/CPU	Multiple APIs with memory queues fragment coordination and caches
Cube refresh worker	Disabled; no generated pre-aggregations	Keep disabled for raw-only mode, or add a separately sized refresh worker after secure pre-aggregation design	It does not execute ordinary raw Trino query capacity
CubeStore router	One stateful router and availability bottleneck	Keep one logical router with durable remote storage, recovery tests, PDB, and supported topology; measure before scaling	Router scaling is not a substitute for API or Trino scaling
CubeStore workers	Two fixed workers, same node as router; unused for raw query acceleration	Spread workers across nodes and size by pre-aggregation ingest/query demand if pre-aggregations are enabled	Extra workers do not speed raw source queries without pre-aggregations
Trino coordinator	One coordinator; query planning and result coordination share one node with worker	Size separately, isolate placement, protect with disruption controls; use supported recovery/routing strategy	Adding coordinator replicas is not a simple horizontal scale mechanism for one cluster
Trino workers	One fixed worker; no admission or fault-tolerant execution	Add resource groups first, then guarded worker scaling with graceful shutdown; consider task retry plus exchange manager for elastic/failure recovery	Gateway or Cube scaling does not create scan/join capacity
Trino Gateway	HPA 1–10, but bypassed by Cube	Route Cube through it only if semantic compute selection/failover is designed and user identity remains correct	Gateway scaling alone cannot execute queries

The safe order is: bound request/result work, eliminate amplification, add admission and truthful queue state, distribute cache/queue state, establish availability, then autoscale the measured saturated layer. A larger fixed profile remains a valid simpler option for low and predictable workspace load. See the shared Trino capacity finding and managed compute lifecycle finding.

Cube's current documentation describes CubeStore as the production cache and queue driver, a production topology of API instances plus a refresh worker and CubeStore, and native request cancellation. See Cube configuration, Cube Core deployment, and Cube client cancellation. Trino documents resource groups, fault-tolerant execution, and graceful worker shutdown.

Ontology PostgreSQL is a one-replica KubeBlocks cluster with one 20 GiB ESSD volume. Its live and source configuration has no backup policy and uses terminationPolicy: Delete. Exact PVC behavior also depends on the storage class reclaim policy, but the model source of truth has no declared backup, restore, recovery-point objective, or recovery-time objective.

The image contains Alembic migrations and the live database reported the image's 7f8a9b0c1d2e head. Neither the image command nor platform-stacks runs or waits for migration. This is not a universal platform behavior: OpenFGA declares a migration job, Airflow enables chart migration jobs, and LTAP declares schema creation. Ontology is a concrete missing instance of a broader need for a standard migration gate across stateful custom services.

Observability boundary

The deployment exposes pod health and standard logs. CubeStore exports metrics, and Trino exposes JMX metrics through ServiceMonitors. The reviewed configuration does not define Ontology-specific metrics or an end-to-end request/query ID. There is no evidenced dashboard, alert, or SLO connecting proposal latency, impact completeness, workbook fan-out, dry-run/load rate, model generation, compiler contexts, Cube queues/cache hits, result bytes, cancellations, Trino queue/run time, or scanned bytes.

At minimum, record project/branch/revision, workbook/tab when applicable, Cube request ID, Trino query ID, terminal state, latency by hop, returned rows/bytes, cache status, and cancellation outcome. Identity and SQL/filters require redaction policy; metrics should use bounded labels rather than raw user, project, branch, workbook, or query identifiers.