magic-data-lifecycle

Routing and orchestration knowledge for data processing tasks. Provides pipeline ordering (load → profile → clean → transform → validate → deliver), skill routing table (which magic-data-* skill handles which operation), quality gating guidance, and checkpoint strategy. Read this skill to understand how data processing phases connect and which skill to invoke for each step. Use when: the task involves multiple data operations, you need to decide which skill handles a specific operation, or the user's request spans multiple processing steps. Trigger keywords: process data, data pipeline, which skill, what order, how to approach this data.

When It Activates

Use this skill when the task involves multiple data processing steps or the user needs help deciding which skill to use. Trigger phrases: process data, data pipeline, which skill, what order, coordinate steps, multi-step, how to approach this data.

Task involves multiple data processing steps that need coordination
You need to decide which magic-data-* skill handles a specific operation
The user's request is vague and spans multiple potential skills
You want pipeline ordering guidance (what comes after loading? when to validate?)

When NOT to Use:

Single, isolated operations — use the specific skill directly (e.g., "just load this file" → magic-data-loading)
User wants the full interactive pipeline with phase tracking → suggest /magic:lifecycle command instead

Quick Facts

Property	Value
Version	2.0.0
Complexity	high
Phase	0
Scripts	0

5-Phase Workflow

The data lifecycle follows a structured 5-phase workflow with PAUSE gates between each phase. Each PAUSE gate requires explicit user approval before proceeding.

Discover → [PAUSE: user reviews findings]
  → Plan → [PAUSE: user approves spec]
    → Execute → [PAUSE: user verifies output]
      → Validate → [PAUSE: user reviews compliance]
        → Deliver

Phase	Skills Involved	Output
Discover	magic-data-loading, magic-data-profiling, magic-data-exploration	Quality score, issue report, patterns
Plan	magic-data-lifecycle (routing)	data-spec.md, processing plan
Execute	magic-data-cleaning, magic-data-transformation, magic-data-synthesis	Cleaned/transformed checkpoints
Validate	magic-data-validation, magic-statistical-analysis	Validation reports, sanity check
Deliver	magic-data-visualization, magic-report-generation, magic-data-transformation	Charts, report, exported data

Tiered Infrastructure

The amount of workspace scaffolding created depends on task complexity:

Tier	When	What Gets Created
Tier 1	Single operation	Just the result — no workspace files
Tier 2	Multi-step pipeline	`workspace_state.md`, `data-spec.md`, `analysis_journal.md`, checkpoints
Tier 3	Multi-dataset projects	Everything in Tier 2 + cross-dataset references and per-dataset subdirs

Skill Routing Table

Operation	Route To
Load file, database, HuggingFace	magic-data-loading
Quality score, distributions, outliers	magic-data-profiling
Fix nulls, normalize, deduplicate	magic-data-cleaning
Schema enforcement, constraint checking	magic-data-validation
Interactive investigation, pattern detection	magic-data-exploration
Pivot, aggregate, merge, derive columns	magic-data-transformation
LLM fill, translate, annotate, enrich	magic-data-synthesis
Hypothesis testing, correlations	magic-statistical-analysis
Charts, plots	magic-data-visualization
Structured report assembly	magic-report-generation

Quality Gating

Gate	Default Threshold
Profiling score before cleaning	≥ 70/100
Cleaning score before analysis	≥ 85/100
Validation pass rate	100% for critical constraints

When a gate fails the agent halts, explains which checks failed, and waits for your instruction.

Scripts

No scripts — this skill provides routing knowledge, not executable code.

Dependencies

pandas numpy

Was this page helpful?