Skip to main content

Notebook Dependency and Network Assessment Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Document the verified Spark Maven/Ivy cache behavior and the missing network boundary between user-controlled notebook workloads and platform-owned databases.

Architecture: Add two focused notebook-execution findings, then link their conclusions into the canonical Spark Operator, notebook architecture, live-evidence, and security pages. Keep platform behavior and trust boundaries in platform-docs while linking implementation work rather than duplicating low-level configuration.

Tech Stack: Markdown, MkDocs, Mermaid, repository link/build checks.

Task 1: Record the two assessment findings​

Files:

  • Create: docs/features/notebook-execution/findings/rel-nb-011-maven-dependency-cache.md

  • Create: docs/features/notebook-execution/findings/sec-nb-003-internal-database-network-boundary.md

  • Modify: docs/features/notebook-execution/index.md

  • Write REL-NB-011 with the controller-side package-resolution lifecycle, the deployed bounded emptyDir mitigation, observed PostgreSQL JDBC package evidence, replica implications, remaining capacity risks, and resolution conditions.

  • Write SEC-NB-003 with verified TCP reachability, the separately user-reported JDBC result, the desired default-deny boundary, required governed paths, compound risk with SEC-NB-002, and explicit unverified items.

  • Register both findings and update the notebook-execution assessment date/evidence summary without replacing existing findings.

  • Check that neither page contains credentials, tokens, passwords, or copied secret values.

Task 2: Update canonical behavior and validation guidance​

Files:

  • Modify: docs/tenant-platform/infra/03-spark-operator.md

  • Modify: docs/features/notebook-execution/architecture.md

  • Modify: docs/features/notebook-execution/live-evidence.md

  • Modify: docs/security/03-data-access.md

  • Explain that Maven coordinates are resolved by the Spark Operator controller before driver creation and that the current cache is per-controller-pod, bounded, and cold after replacement.

  • Add the package repository/cache and internal-service trust boundary to the notebook architecture, clearly separating current evidence from the target security posture.

  • Add NB-S023 for safe package-resolution verification and NB-S024 for non-destructive internal-service boundary verification.

  • Add a concise security cross-link to SEC-NB-003 without duplicating the finding.

Task 3: Validate and prepare the pull request​

Files:

  • Verify all files changed by Tasks 1 and 2.

  • Run git diff --check.

  • Run the repository's documented docs build or validation command from a clean dependency state where practical.

  • Verify all new relative links resolve and the new IDs are unique.

  • Search changed content for credential-like values and unsupported claims; retain only sanitized evidence.

  • Review the complete diff against the approved design, commit the implementation, push docs/notebook-ivy-network-assessment, and open a pull request against main.