Process Factory Architecture v1.0

Overview

The Process Factory Engine replaces ad-hoc skill invocation with tracked, stateful DAG execution. Every skill, ingest job, or analysis task becomes a Block — a universal execution unit with typed inputs, outputs, dependencies, gates, and thresholds.

Core Principles

• Config over code — Pipeline topology defined in JSON, not hardcoded
• Stateless scheduler — Given current block states, computes readiness. No memory needed.
• One schema, three patterns — Block v2 handles DAG batch, stream/event, and analytical lens
• Human-in-the-loop — Gates before or after any block. Approve/reject with feedback.
• Threshold enforcement — Auto-retry below minimum, fail after max attempts
• Modular architecture — Engine tracks state. Claude Code/FORGE executes skills. Dashboard observes.

Block v2 Schema

The universal execution unit. Every skill, ingest job, or analysis task = one block. 11 field groups covering all three execution patterns.

{
  // ── Identity ──
  id:              string       // "soul-extractor"
  name:            string       // "Soul Extractor"
  layer:           string       // "L2"
  type:            enum         // extract|validate|build|orchestrate|research|ingest

  // ── Execution ──
  skill:           string       // maps to SKILL.md
  model:           string|null  // "opus"|"sonnet"|null (inherit)
  timeout:         number       // 300000 (ms)
  runtime:         enum         // forge|claude-cli|mastery-api|nowpage

  // ── Trigger ──
  trigger: {
    type:          enum         // dependency|cron|webhook|event|manual
    cron:          string|null  // "0 6 * * *" (Pattern 2)
    event:         string|null  // "entity.created(gate=2)" (Pattern 2)
    webhook:       string|null  // "/api/ingest/youtube" (Pattern 2)
  }

  // ── Inputs ──
  inputs: {
    depends:       [{block, key}]   // wiring from other blocks
    files:         [glob]           // workspace file patterns
    env:           [string]         // required env vars
    config:        {k:v}            // runtime config overrides
  }

  // ── Outputs ──
  outputs: {
    artifacts:     [{key, path, format}]
    metrics:       [{key, type}]
    publish:       [{domain, slug}]|null
  }

  // ── Execution Control ──
  execution: {
    parallel_group: string|null     // "L2-extractors"
    human_gate:     enum            // before|after|none
    threshold:      {metric, min}|null
    retryable:      boolean
    max_retries:    number          // 2
  }

  // ── Lifecycle (Pattern 1: one-shot, 2: daemon, 3: on-demand) ──
  lifecycle: { mode: enum }         // one-shot|daemon|on-demand

  // ── Scope ──
  scope: {
    gate:          1|2|3|4|null     // Data Gate tier
    run_params:    {k:v}            // per-run overrides
  }

  // ── Lens (Pattern 3 only) ──
  lens: {
    perspective:   string|null      // "positioning"|"gap_analysis"
    aggregates:    [string]|null    // ["entity_id"]
    feedback_loop: boolean
  }

  // ── Record State (Pattern 2 only) ──
  record_state: {
    track_per:     string|null      // "content_id"|"entity_id"
    states:        [string]         // ["raw","annotated","embedded","scored"]
  }

  // ── Runtime State (engine-managed, not in config) ──
  state: {
    status:        enum             // pending|ready|running|done|failed|blocked|waiting
    started_at:    timestamp|null
    ended_at:      timestamp|null
    score:         number|null
    error:         string|null
    attempts:      number
  }
}

Design Decisions

• 11 field groups — covers all three patterns with one definition
• Null fields for inactive patterns — Pattern 1 blocks have lens: null and record_state: null
• Runtime state is engine-managed — config defines the block, engine manages the state
• Typed wiring — depends: [{block: "soul-extractor", key: "soul-json"}] creates explicit data contracts

Three Execution Patterns

All use the same Block v2 schema. Only the scheduler behavior differs.

Clone Factory DAG — 18 Blocks

The complete pipeline for cloning an expert's IP into an AI coaching OS.

Legend

Key Properties

• 3 Parallel Groups: L0-recon (3 blocks), L2-extractors (5 blocks), L3-builders (4 blocks)
• 3 Human Gates: After expert-framework-creator, gap-analyzer, clone-tester
• 1 Threshold: clone-tester must score ≥ 85% or fail
• Max retries: 2 per block (configurable)
• Total blocks: 18 (no cycles, valid DAG)

Data Flow Map

Every block's typed inputs and outputs, showing how data flows through the pipeline.

Module ↔ Skill Mapping (PRD 4)

How PRD 4's 9 extraction modules map to our 18 skills. Coverage: 9/9 mapped.

Execution Timeline

Parallel groups and human gates shown on a layer-by-layer timeline.

LAYER     BLOCKS                           GATE          PARALLEL?
────────  ─────────────────────────────    ──────────    ─────────
L0        deep-research                                  YES (3)
          expert-recon                                   (L0-recon)
          masterybook-sync

L0.25     expert-framework-creator         AFTER ⛔      Sequential

L0.5      demo-compiler                                  Sequential

L1        rubric-builder                                 Sequential

L2        soul-extractor                                 YES (5)
          voice-extractor                                (L2-extractors)
          framework-extractor
          resource-extractor
          offer-extractor

L2.5      gap-analyzer                     AFTER ⛔      Sequential

L3        clone-compiler                                 YES (4)
          lead-magnet-builder                            (L3-builders)
          onboarding-builder
          design-system-extractor

L3.5      clone-tester (85% min)           AFTER ⛔      Sequential

L4        boarding-orchestrator                          Sequential
      

Timing Estimates (based on Run #1)

• L0 parallel group: ~10 min wall clock (deep-research is the bottleneck)
• L0.25 framework: ~15 min (3-path convergence, worth the cost)
• Human gate 1: Variable (expert reviews framework)
• L2 parallel group: ~20 min wall clock (framework-extractor is heaviest)
• Human gate 2: Variable (product lead reviews gaps)
• L3 parallel group: ~25 min wall clock (clone-compiler is heaviest)
• L3.5 testing: ~30 min (27 scenarios)
• Total machine time: ~2 hours (excluding human gates)

Intelligence Gathering Pattern (Pattern 3)

How pre-meeting analytical lenses work — future capability for PRD 5.

Block v2 Fields Used

• lens.perspective — which analytical angle this block takes
• lens.aggregates — what dimension to roll up on (entity_id)
• lens.feedback_loop — whether to collect post-execution quality signals
• lifecycle.mode: "on-demand" — runs when triggered, not continuously

Connection to deep-research

Pattern 3 blocks can delegate heavy research to the existing deep-research skill (via Perplexity Sonar), then apply their specific analytical lens to the raw results. The research infrastructure is shared; only the interpretation differs.

PRD Reconciliation

What matches, what's missing, what's deferred.

Extensions Needed for Future Patterns

Critical rule: All extensions are ADDITIVE. They add new scheduler behaviors and field consumers. They do NOT change Block v2 or the existing DAG scheduler.

Build Order

Session Continuity Protocol

If a session dies, the next session can reconstruct full context from these durable artifacts:

Key Files