Data Catalog

The Data Catalog lets workspace users discover catalog, schema, table, and column names and then inspect data, permissions, metadata, lineage, history, and quality information. This is the canonical assessment of that capability.

See Architecture for system boundaries and Evidence for verification and reproduction steps.

Status

Field	Current value
Availability	Available in realm gj76anadpsc, workspace w-t5e2kcxjd
Supported user-facing catalog	delta only in the assessed deployment
Assessment coverage	Partial — source and deployed configuration are verified; signed-in UI and restricted-role journeys remain open
Kubernetes context	dev-north-alicloud
Last verified	2 September 2026
Tracking	platform-docs #44; component validation is not recorded

Who determines the catalog list?

No single component currently owns the complete contract.

Decision	Authority
Catalogs that Trino can query	Platform deployment configuration. Each /etc/trino/catalog/*.properties file defines one Trino catalog and connector.
Top-level entries shown by Cogrion	Superset's database registry as visible to the exchanged user token. Oqullus calls GET /superset/api/v1/database; it does not call SHOW CATALOGS.
Schemas, tables, and columns beneath an entry	Superset database APIs backed by the registered connection; the current delta connection introspects Trino, where Ranger can filter or deny metadata operations if the effective user is preserved.
Table/column descriptions, tags, structured properties, and lineage	DataHub, addressed with dataset URNs. DataHub does not determine the top-level navigation list. Catalog/schema description and tag controls are not connected to this path.
Query-time data access and the Permissions tab	Ranger policies; the BFF restricts all reviewed Ranger catalog routes to platform_admin.
Browser tree and expansion state	Oqullus memory for the current page session. It is a cache, not an authority.

The effective product contract is therefore the Superset registrations created by the supported platform stack and verified against the corresponding Trino catalog. Adding an arbitrary Superset database may make it appear in the UI, but does not make all Catalog features compatible or supported.

Superset itself supports Trino's catalog level and exposes catalog discovery for catalog-capable engines. The missing level is specific to the Cogrion integration: Oqullus treats each Superset database connection as a catalog node and proceeds directly to that connection's schemas. For the current single-Trino design, the UI may hide the connection level, but the visible delta node should still come from Trino catalog discovery rather than from a Superset connection that happens to be named delta.

Catalog landing behavior

/admin/catalog should be a browse entry point, not a placeholder Quick Access page. With one authorized catalog, it should open that catalog's explorer directly. With multiple authorized catalogs, it should show a searchable catalog directory. Zero authorized catalogs and discovery failure must have different states so denial is not confused with outage or absence.

The current Recents tab does not implement recency. It displays the same complete list as Catalogs, and every Last viewed value is hard-coded to N/A. Remove Quick Access, Recents, and Last viewed until durable user- and workspace-scoped visit tracking exists.

Supported catalogs

Name	Deployed Trino connector	Registered in Superset	Shown in Catalog	Status
delta	delta_lake	Yes	Yes	Supported for this assessment
hive	hive	No	No	Deployed but unsupported; no current consumer was identified
tpch	tpch	No	No	Synthetic Trino test data; not a product data catalog
tpcds	tpcds	No	No	Synthetic Trino test data; not a product data catalog
iceberg	Not deployed	No	No	Legacy bundle configuration only; not supported in the current KCL path or assessed deployment
Databricks, external SQL, and connector sources	Separate connector/preview flows	No verified Catalog registration	No	Outside this Data Catalog support contract

AWS and workspaces other than w-t5e2kcxjd are not runtime-verified. Current KCL also seeds only the delta Superset registration, but that is source evidence rather than a cross-environment support commitment.

System schemas

information_schema is virtual Trino metadata needed for discovery and client metadata queries. It should be readable only through Ranger-filtered metadata operations and hidden from normal Catalog browsing by default. It is not a governance target: users must not be offered descriptions, tags, ownership, quality rules, sampling, or history actions for it.

cogrion_system contains product-internal tables for data quality, FAQs, PII scanning, and recommendations. Hide it from normal Catalog and SQL Editor discovery and deny direct user SQL by default. Product endpoints should use separate least-privilege service identities after authorizing the caller; users must not receive broad access merely to make an internal feature work.

Actors and access conditions

The frontend exposes the Catalog route to platform_admin, tenant_admin, data_engineer, ml_engineer, business_user, and data_steward. It excludes data_analyst. The product rationale for that distinction is not documented.

Opening the page is separate from accessing data. Superset permissions determine whether its registered delta connection appears. Ranger does not determine that top-level list. Deeper metadata operations and SQL pass through Trino, where Ranger can enforce the effective user. Metadata mutations instead go through the BFF to DataHub. Current source does not consistently hide mutation controls for roles that the backend will reject.

Critical journeys

Journey	Current evidence	Required verification
List catalogs	Source and deployed Superset metadata confirm delta.	Capture the signed-in Catalog list and its network response.
Expand schema, table, and columns	Lazy Superset-backed calls are source-mapped.	Verify normal, empty, unauthorized, dependency-failure, and retry behavior.
Inspect sample data	Opening a table automatically starts a limited query through Superset and Trino.	Verify Ranger denial, partial failure, data types, large values, and visible error handling.
Inspect history	A connector-specific table$history query runs automatically.	Confirm supported Delta behavior and the response for unsupported connectors or tables.
Edit table and column descriptions and tags	DataHub mutations and on-demand table-schema ingestion exist.	Verify permitted and denied roles, audit identity, propagation, conflict, and recovery. Catalog/schema metadata is not supported.
Inspect lineage	DataHub lineage APIs exist.	Verify canonical URN construction and actual lineage for a known table.
Manage permissions	Ranger list/grant/revoke routes require platform_admin.	Verify role-aware controls, inherited scope, audit, denial, and query enforcement.
Search	Client search loads unloaded catalogs and schemas, then filters locally.	Establish request, latency, table-count, and browser-memory boundaries.
Upload CSV or Excel	Catalog offers upload to a registered Superset database.	Verify authorization, destination, collision, size, cleanup, and partial failure.

Findings

Use this table as the assessment register. In each finding, Implementation issue identifies the repository where the code change is tracked and Component involvement explains any additional repository or runtime role. These fields are not person or team assignments. Use Live evidence for the complete execution scenarios.

ID	Finding	Evidence	Impact
ARCH-CAT-001	Separate connection routing from catalog identity	Source + deployed configuration	High
ARCH-CAT-002	Use one catalog-tree presentation across Catalog and SQL Editor	Source-verified	Medium
ARCH-CAT-003	Replace browser-authored DQ SQL with a resource API	Source + captured live failure	High
COST-CAT-001	Load table-detail tabs on demand	Source + live failures	High
DATA-CAT-001	Use one canonical DataHub dataset identity	Source-verified	High
DATA-CAT-002	Declare how Catalog metadata reaches DataHub	Source-verified; runtime coverage pending	High
DATA-CAT-003	Hide catalog and schema fields without an authority	Source-verified	High
DATA-CAT-004	Preserve table format in shared-HMS discovery	Source-verified; mixed-format fixture pending	High
GOV-CAT-001	Separate system schemas from governed user data	Source-verified; role visibility pending	High
LIFE-CAT-001	Assign or retire the deployed Hive connector	Source + partial usage history	Medium
OPS-CAT-001	Make DQ initialization observable and durable	Deployed failure observed	Critical
PERF-CAT-001	Bound Catalog search without loading the entire tree	Source-verified; scale pending	Medium
PERF-CAT-002	Define bounded Sample and History execution	Source-verified; benchmarks pending	High
PROD-CAT-001	Remove unsupported catalog lifecycle controls	Source-verified	Medium
PROD-CAT-002	Hide unsupported adjacent-product controls	Source + live observation	Medium
PROD-CAT-003	Replace fake Recents with a useful Catalog landing	Source-verified	Medium
PROD-CAT-004	Remove non-persistent catalog and schema metadata controls	Source-verified	High
REL-CAT-001	Distinguish discovery failure from an empty node	Source-verified	High
REL-CAT-002	Retry only transient, safe Catalog failures	Source + repeated live failures	High
SEC-CAT-001	Authorize DataHub mutations per actor and resource	Source-verified; security review pending	Critical
SEC-CAT-002	Align permission controls with verified authorization	Source-verified; role matrix pending	High
SEC-CAT-003	Preserve end-user identity through Catalog discovery	Architecture verified; runtime identity pending	High
SEC-CAT-004	Make system-schema policy deterministic	Source-verified; deployed policies pending	High
UX-CAT-001	Do not present local rename as a durable mutation	Source-verified; live reproduction pending	High
UX-CAT-002	Use consistent object-type icons	Source + live observation	Medium
UX-CAT-003	Remove the artificial Warehouse hierarchy	Source + product review	Low
UX-CAT-004	Provide one consistent Upload data action	Source + live observation	Medium
UX-CAT-005	Remove the inert Catalog metastore setting	Source-verified	Medium
UX-CAT-006	Keep display timezone consistent with query semantics	Source + product review	High
UX-CAT-007	Preserve typed Spark lineage	Source + product review	High

Implementation subissues

All remediation remains unassigned and is grouped under platform-docs #44.

Finding	Issue	Finding	Issue
ARCH-CAT-001	oqullus #527	ARCH-CAT-002	oqullus #528
ARCH-CAT-003	bff-api #234	COST-CAT-001	oqullus #533
DATA-CAT-001	oqullus #529	DATA-CAT-002	platform-stacks #674
DATA-CAT-003	oqullus #536	GOV-CAT-001	bff-api #236
DATA-CAT-004	bff-api #238
LIFE-CAT-001	platform-stacks #675	OPS-CAT-001	platform-stacks #672
PERF-CAT-001	oqullus #532	PERF-CAT-002	bff-api #233
PROD-CAT-001	oqullus #537	PROD-CAT-002	oqullus #538
PROD-CAT-003	oqullus #539	PROD-CAT-004	oqullus #540
REL-CAT-001	oqullus #531	REL-CAT-002	oqullus #534
SEC-CAT-001	bff-api #232	SEC-CAT-002	oqullus #535
SEC-CAT-003	bff-api #235	SEC-CAT-004	platform-stacks #673
UX-CAT-001	oqullus #530	UX-CAT-002	oqullus #541
UX-CAT-003	oqullus #542	UX-CAT-004	oqullus #543
UX-CAT-005	oqullus #544	UX-CAT-006	oqullus #545
UX-CAT-007	bff-api #237

Detailed rationale and reproduction steps are in Evidence. Findings are observations, not accepted implementation commitments. Missing evidence stays open, and severity and disposition require review.

Boundaries and workarounds

• Treat delta as the only supported top-level catalog in the assessed workspace. Do not register another Superset database merely to make it appear.
• Do not remove hive based only on Superset history. First inventory non-Delta Hive tables, all-client Trino audit history, and Ranger policies, then perform a monitored deny or disable canary.
• Use the refresh control after platform-side catalog changes; there is no automatic reconciliation or durable browser cache.
• Do not rely on tree Rename, Create Catalog, Create Schema, or Set Owner as durable operations.
• Ignore schema Owner and Created at values; remove them rather than treating N/A and the hard-coded date as metadata. Keep the corresponding Catalog fields hidden until an authoritative, persistent source exists.
• Do not use catalog/schema Description or Tags controls. Durable DataHub-backed metadata is currently implemented only for tables and columns.
• Hide information_schema from normal Catalog navigation and block governance mutations for it. Permit only Ranger-filtered read-only metadata access.
• Hide and deny direct user access to cogrion_system; use scoped service identities for authorized internal product operations.
• Ignore the Catalog gear's metastore value; it is hard-coded and not a setting. Remove it until an admin-only live diagnostic capability exists.
• Do not grant users direct access to cogrion_system to make the current DQ loader succeed. Load DQ state through a resource-oriented BFF endpoint and fetch it only when Quality is first opened.
• Treat DQ as unavailable until initialization verifies its schema, tables, Delta files, and seed rows through the same Trino path used at runtime. A running watcher is not evidence that DQ is ready.
• Treat Sample as an arbitrary preview, not a random or representative sample. For a slow table, use SQL Editor with explicit columns and an approved partition predicate until bounded preview execution exists.
• Treat History timestamps as instants supplied by Delta $history. Use UTC with an explicit offset; the current Local selection is display-only and resets on reload. Do not infer query/filter semantics from it.
• Delta Sharing, Clean Rooms, and External Data are not available; their disabled controls do not indicate a role, configuration, or workspace issue.
• Treat Recents and Last viewed as non-functional presentation. Use Catalogs or the navigation tree; no reliable visit history currently exists.
• Use SQL Editor with an explicit fully qualified name to diagnose whether a Trino catalog exists independently of its Catalog-page registration.
• Do not treat visibility of the top-level delta entry as proof of Ranger authorization. Verify access by expanding the object or executing an approved query until SEC-CAT-003 is closed.
• No supported maximum is documented for catalogs, schemas, tables, columns, search fan-out, sample cell size, uploads, lineage graph size, or tag count.

Review status

Product, data-platform, metadata-governance, security, and operations owners have not yet validated this assessment. Signed-in UI evidence and restricted role tests are pending; those gaps are not evidence of support.