magic-data-synthesis

Synthesize, generate, and transform data using LLM-based operations via the DataDesigner engine. The primary engine for all LLM-powered data generation tasks.

When It Activates

Use this skill when generating data using LLM or creating synthetic examples. Trigger phrases: synthesize, generate, fill missing, translate, annotate, enrich, augment data, create new examples, DataDesigner, LLM generation.

• Columns have missing values, sentinels ("X", "N/A", "TBD"), or placeholders needing contextual generation
• Format conversion (HTML→markdown), translation, annotation, labeling, summarization
• Structured field extraction from unstructured text into multiple columns
• Reference join leaves gaps → LLM fills remaining (use enrich_from_reference.py first)

When NOT to Use: Rule-based fixes (regex, type casting, dedup) → magic-data-cleaning. Reshaping, joins, aggregation → magic-data-transformation. Schema enforcement → magic-data-validation. If a Python function can produce correct output for every case, use programmatic generation instead of DataDesigner.

Quick Facts

Tags

data-science synthesis generation llm transformation enrichment annotation

DataDesigner: Primary Synthesis Engine

DataDesigner is the primary engine for all LLM-powered data generation. It replaces the legacy batch_synthesize.py approach with a config-driven model that provides preview gates, cost estimation, and automated quality control.

Key Differences from Legacy Approach

Config-Driven Generation

DataDesigner configurations are Python files exposing a load_config_builder() function that returns a DataDesignerConfigBuilder. The agent writes this config adapted to the task:

# File: workspace/configs/product_recipe.py
from data_designer.config.config_builder import DataDesignerConfigBuilder, ModelConfig
from data_designer.config.models import ChatCompletionInferenceParams
from data_designer.config.seed_source_types import LocalFileSeedSource


def load_config_builder() -> DataDesignerConfigBuilder:
    # Model config — change provider/model/alias for your endpoint
    model = ModelConfig(
        alias="gen-model",
        model="gemini-2.5-flash-lite",
        inference_parameters=ChatCompletionInferenceParams(
            max_parallel_requests=4, temperature=0.7, max_tokens=256,
        ),
        provider="gemini",  # must exist in ~/.data-designer/model_providers.yaml
        skip_health_check=True,
    )
    builder = DataDesignerConfigBuilder()
    builder.add_model_config(model)
    # Seed dataset — DataDesigner generates a value for every seed row
    builder.with_seed_dataset(LocalFileSeedSource(path="data/input/products.csv"))
    # LLM-generated column — {{ col }} references seed columns via Jinja
    builder.add_column(
        column_type="llm-text",
        name="product_description",
        prompt="Write a 2-sentence product description for {{ product_name }} in category {{ category }}.",
        model_alias="gen-model",
    )
    return builder

Preview Gate (Required)

Always validate and preview before full generation. Validation catches config errors before any API calls; the preview gate shows sample output and a cost estimate before committing. Run the recipe through the data-designer CLI:

# 1. Validate the config (no API calls) — expect "✅ Configuration is valid"
data-designer validate workspace/configs/product_recipe.py

# 2. Preview: generate 5 sample rows and save them for review
data-designer preview workspace/configs/product_recipe.py \
  --num-records 5 --save-results --artifact-path workspace/artifacts/preview

# 3. Only after the preview looks correct, run the full generation
data-designer create workspace/configs/product_recipe.py \
  --num-records 1000 --artifact-path workspace/artifacts/run_001

The preview gate is mandatory — even in autonomous mode. Estimate cost from the preview sample (estimate_from_preview()) before calling create(): local models are $0; for cloud models, extrapolate the preview token counts to the target row count.

LLMJudgeColumnConfig — Automated Quality Scoring

LLMJudgeColumnConfig adds an LLM judge as its own column. Each judge carries one or more Score rubrics (all evaluated in a single LLM call per row), and the scores are written back as a nested-dict column named {name}_judge_result:

import data_designer as dd

builder.add_column(
    dd.LLMJudgeColumnConfig(
        name="translation",
        model_alias="judge-model",  # ideally a separate, stronger judge model
        prompt="Evaluate the translation of {{ text }} into {{ translation }}.",
        scores=[
            dd.Score(
                name="accuracy",
                description="Is the translation accurate and natural-sounding?",
                options={0: "No", 1: "Partial", 2: "Yes"},
            ),
        ],
    )
)

Scores land in the translation_judge_result column shaped {rubric: {score, reasoning}}. Filter or regenerate rows by thresholding those scores after generation (e.g. df["translation_judge_result"].apply(lambda x: x["accuracy"]["score"] >= 2)) — there is no separate failed_rows attribute.

Scripts

Scriptable Tools (call directly or read + adapt)

Reference Implementations (read patterns, write custom code)

Cost Estimation

Always estimate cost before full generation. The estimate is derived from the token counts observed in the preview sample. Run a small preview first, then extrapolate to the target row count:

# Preview 10 rows and save the sample + token accounting
data-designer preview workspace/configs/product_recipe.py \
  --num-records 10 --save-results --artifact-path workspace/artifacts/preview

Read the preview artifact's token counts and multiply by the target row count (estimate_from_preview() does this from the saved preview). Cost scales roughly linearly with row count for most column types. Local models are $0; thinking models (extended-reasoning models) cost significantly more — use them only for complex reasoning tasks.

Supported Operations

Dependencies

pandas numpy data-designer tiktoken