Maintaining the Recipe Language

This page is for maintainers extending recipe language 2.0.0 or preparing a future language version. For the current data flow and validation stages, see Recipe Language 2 Architecture. Exact fields and variants are listed in the Recipe Language 2.0 Reference.

Terminology and ownership

Step definition

Declarative recipe data represented by a strict, frozen Pydantic model in pypts.recipe_language. StepDefinition is the discriminated union of every step users may place in YAML.

Validated recipe definition

The aggregate pypts.recipe_language.Recipe returned by ParseResult.require_recipe() after YAML, structural, and semantic validation succeeds.

Executable step

A runtime object implemented in pypts.steps. It owns behavior such as _step() but does not define the authored recipe schema.

Runtime-only step

An internal operation that cannot be written directly in YAML. For example, IndexedStep is created when a validated direct input has indexed set.

The Pydantic models are the only structural definition of the language. JSON Schema, generated reference documentation, YamVIEW selectors, and form fields all derive from them. Do not add a second list of supported steps, mappings, required fields, defaults, or aliases in a consumer.

Runtime dispatch and dependency direction

Step.build_step() in pypts.recipe is the single boundary between a validated step definition and an executable step:

recipe_language.StepDefinition
             |
             v
    Step.build_step()
             |
             v
    STEP_TYPE_REGISTRY
             |
             v
    executable class in steps.py

STEP_TYPE_REGISTRY is intentionally beside the factory. It answers only: “Which executable class implements this validated discriminator?” It does not own fields, defaults, validation rules, or the set exposed by JSON Schema.

Moving the registry into steps.py would reverse the existing dependency direction: executable classes in steps.py derive from or interact with runtime types defined in recipe.py. Keeping dispatch in recipe.py avoids introducing a runtime adapter or circular registration mechanism. A completeness test requires the registry keys to equal the discriminators in STEP_DEFINITION_MODELS exactly.

Adding a step definition

  1. Add the strict Pydantic definition in recipe_language.py and include it in the StepDefinition discriminated union. Use the exact canonical steptype literal; do not add normalization or lowercase aliases.

  2. Implement the executable class in steps.py. Keep recipe-structure validation in the definition model or parser rather than its constructor.

  3. Add one discriminator-to-class entry to STEP_TYPE_REGISTRY beside Step.build_step(). Do not register runtime-only wrappers.

  4. Add parse/dump/reparse coverage, runtime construction assertions, and a behavioral execution test. The registry-completeness test must remain exact.

  5. Build the documentation and review the generated schema, reference section, and YamVIEW selector. These consumers should update without hardcoded lists.

Adding fields or mapping variants

Put a field’s type, required status, default, description, examples, aliases, and serialization behavior on its Pydantic declaration. A rule concerning one object belongs in a model validator. A rule requiring other sequences, sibling values, reference resolution, or execution order belongs in the parser’s aggregate semantic pass so it can produce a structured source-span diagnostic.

For a new input or output mapping, add its definition to the corresponding discriminated union and test parse, canonical dump, reparse, generated schema, reference rendering, and YamVIEW mapping rows. Runtime mapping behavior must also receive a focused execution test.

Preparing a language version upgrade

A version change is a deliberate compatibility decision, not a normalization shortcut. Before changing the header’s recipe_version literal:

  • define the supported version policy and whether migration is a separate release phase;

  • document removed, renamed, and newly required fields and discriminators;

  • update parser diagnostics for missing, legacy, and unsupported versions;

  • update canonical examples, fixtures, schema expectations, and release notes;

  • decide explicitly whether multiple versions have separate models and runtime paths; never silently coerce one version into another;

  • verify that invalid or legacy documents cannot construct executable state;

  • review generated JSON Schema and reference artifacts before release.

Documentation and verification

Sphinx generates the JSON Schema first and renders the human reference from that exact file. Do not hand-edit files below docs/source/_generated. Run:

python -m pypts.recipe_artifacts
python -m pytest tests
python -m ruff check src tests
python -m sphinx -W -b html docs/source docs/_build/html

At minimum, a recipe-language change must pass deterministic artifact tests, all discriminator and runtime-registry completeness tests, the full unit and functional suite, and Sphinx with warnings treated as errors.