REL-WF-003 — Reconcile Git publication with workflow database state
| Field | Value |
|---|---|
| Status | Open; source-verified, fault injection pending |
| Architecture state | Deployed publication flow; authorization outbox inactive with OpenFGA |
| Severity | High |
| Affected surface | Create, update, rename, delete, publication retry, and orphan recovery |
| Components | sparqd/workflow-backend, DAG Git/publisher, database, Airflow |
| Associated issue | Not opened |
Where the problem appears
Inject failures between successful Git publication or deletion and the following workflow database commit.
Concrete observed case
Create and update publish DAG artifacts before committing the workflow record. Delete removes artifacts before deleting the record. If the later database commit fails, Git and product state can disagree. Workflow-backend source contains an authorization-outbox pattern, but it is tied to inactive OpenFGA integration and does not cover DAG publication.
Expected condition
Persist a publication/deletion intent with the workflow transaction, execute it idempotently in a worker, and reconcile desired revision against Git and Airflow until convergence.
Cause and evidence
- Git mutation and database commit are separate external transactions.
- The request rolls back database work when Git itself fails, but cannot roll back a Git push after a later database failure.
- Source demonstrates a durable authorization-outbox pattern, but its OpenFGA consumer is inactive and publication does not use the pattern.
Why it matters
Failures can create orphan DAGs, missing DAGs for existing records, or revisions that run without matching product state.
Existing controls and workarounds
After any create/update/delete 5xx, compare the workflow record, Git artifact, and Airflow DAG before retrying. Escalate mismatches rather than guessing which side won.
Alternatives considered
- Recommended: transactional publication outbox plus idempotent worker and reconciler.
- Git-first with compensating rollback — difficult because compensation can also fail and concurrent revisions race.
- Database-first synchronous publication — makes desired state durable but still requires asynchronous retry and reconciliation.
Component involvement
- sparqd/workflow-backend owns desired state and publication intent.
- A publication worker owns Git convergence.
- Operations observes workflow UUID, revision, commit SHA, attempt, and Airflow result.
Reproduction and verification
Follow WF-S017.
Resolution conditions
Close when fault injection across create, update, rename, and delete proves idempotent convergence without orphaned or missing runnable DAGs.
Validation and blockers
Workflow-backend, Git repository, Airflow, database, and operations owners must validate.