DATA-ML-002 — Define the synthetic-data contract
| Field | Value |
|---|---|
| Status | Open finding; source-verified; live generation pending |
| Severity | High |
| Components | sparqd/agent-based-ml-platform, sparqd/bff-api, sparqd/oqullus, object storage |
| Associated issues | Not opened pending finding review |
Concrete observed case
Synthetic Data accepts a CSV, optional target, and multiplier, polls a separate run, and displays quality and history. The saved "generated dataset" is actually a shuffled augmentation containing all source rows plus generated rows. This retains the source data's classification and privacy risk.
The backend caps input at 200 MB and combined output at 500,000 rows and accepts multipliers from 0.1 to 5. Oqullus accepts only whole numbers of at least 1 and does not state the upper limits. Scores below 0.65 produce a warning, not a failure. If quality evaluation itself fails, the run may still complete with a zero score and warning. Stale backend documentation instead says low quality is rejected.
The backend has an output download operation, but BFF and Oqullus do not expose it. Oqullus displays the internal storage path as the result. The generator also does not observe cancellation before writing output and terminal state.
Expected condition
Name the output augmented dataset or offer an explicit synthetic-only mode. Show and enforce the same limits at every layer. Define a quality policy that distinguishes evaluation failure from a measured low score. Provide an authorized expiring download and preserve source, synthesizer, seed, generation, deduplication, evaluation, classification, retention, and deletion lineage.
Why it matters
Users may treat an augmented file as de-identified synthetic data even though it contains original records. Warning-only or unavailable quality cannot support a privacy or fitness claim, and an internal path is neither a usable nor safe delivery contract.
Existing controls and workarounds
The generator removes exact row-hash matches from generated rows, calculates quality metrics when possible, reports warnings, and enforces backend size and row limits. Operators can retrieve an object manually, but that bypasses the intended user authorization and audit path.
Alternatives considered
- Recommended: explicit augmented versus synthetic-only products with distinct quality/privacy claims, lineage, authorized retrieval, and retention.
- Keep warning-only quality for exploratory use, but label output unsuitable for privacy or production decisions and require acknowledgement.
- Treat a numeric quality threshold as proof of anonymization — rejected; utility metrics do not establish disclosure risk.
Component involvement
- AutoML owns generation semantics, reproducibility, evaluation state, and artifact metadata.
- BFF owns authorized status, history, and download mediation.
- Oqullus displays limits, composition, quality state, warnings, and retrieval.
- Object-storage policy owns classification, retention, and deletion.
Reproduction and verification
Follow ML-S012.
Resolution conditions
Close when users can distinguish augmented from synthetic-only data, evaluation failure cannot look like acceptable quality, downloads are authorized, cancellation cannot publish output or overwrite terminal state, and privacy, lineage, reproducibility, retention, and deletion are verified.
Validation and blockers
The source behavior is confirmed. Representative generation, privacy review, quality acceptance criteria, and artifact lifecycle evidence remain pending.