Engine workflow

FalconSmith is not a one-shot generator. It is a reconciliation engine that continuously enforces structural and semantic consistency within a generated library.

This page describes the intentional, developer-centric execution workflow that governs how a FalconSmith library is evolved, rebuilt, and verified over time.

The workflow is designed to feel like a natural extension of how developers already work: write Python, reconcile the namespace, test and iterate; introduce native code when needed, rebuild and expose it explicitly, switch implementations freely, and continue evolving the system without breaking structure or visibility.

You write Python, run forge namespace, import and test. When performance matters, you drop native code into _native, run forge smithery, expose it, and keep going. When it breaks, you replace it - not the workflow.

The problem FalconSmith solves

In long-lived codebases, most failures do not originate in algorithms. They emerge from structural drift.

• Namespaces that no longer reflect the filesystem
• Partially exposed or inconsistently wired functionality
• Native code that compiles but is misintegrated
• Documentation that no longer reflects behavior
• Implicit assumptions known only to original authors

FalconSmith addresses this by enforcing an explicit and repeatable workflow that keeps structure, native code, interfaces, and documentation in sync.

These failures rarely surface immediately - they accumulate silently until maintenance, onboarding, or extension becomes risky.

The FalconSmith engine model

FalconSmith operates as a reconciliation engine. Each engine command is responsible for enforcing correctness in a single, well-defined dimension of the system.

Rather than hiding behavior behind implicit build steps, FalconSmith exposes explicit entry points for each responsibility.

The engine is composed of five primary components:

• Namespace - Python API surface and visibility
• Smithery - native build, exposure, and inventory
• Tree - materialized structure and exposure layout
• Docs - documentation parity and API truth
• Runtime state - the approved runtime, drift detection, and controlled evolution

Each component can be executed independently, but all are designed to converge on a single, verifiable system state. No phase relies on implicit execution or runtime side effects. All operations are explicitly triggered.

Namespace reconciliation: forge namespace

FalconSmith treats the Python namespace as a generated artifact, not a manually curated structure.

The namespace phase reconciles filesystem layout with the public API surface.

• Scans project structure
• Regenerates __init__.py and __all__
• Ensures import paths reflect actual filesystem layout
• Removes stale or invalid exports

This eliminates a common failure mode where functionality exists on disk but is invisible or inconsistently exposed to users.

Manual namespace maintenance is intentionally avoided.

Native evolution: forge smithery

Native Smithery is the engine responsible for the full lifecycle of native code inside a FalconSmith project.

It governs compilation, exposure, inventory, and verification as a single, explicit operation.

• Scans native source trees
• Generates and updates build configuration
• Compiles native extensions deterministically
• Analyzes compiled artifacts during controlled build steps
• Materializes the Python-facing native API surface
• Updates native inventories and manifests

Native code is never compiled implicitly. All native evolution occurs through this explicit phase.

If Smithery fails, native code is not partially integrated or exposed.

Exposure and structure materialization: forge tree

The tree phase materializes the structural truth of the project.

It generates and verifies the explicit filesystem representation of exposed Python and native components.

• Provides a materialized view of exposure structure
• Reflects the current validated state of native and Python exposure
• Supports inspection and verification of structural consistency

The generated tree is not a visualization - it is a verified artifact used by other engine components.

Tree generation never performs compilation or imports.

Documentation parity: forge docs

Documentation is treated as a generated view of the system, not a parallel source of truth.

The documentation phase derives its content directly from Python code, exposure manifests, and validated inventory data.

• Scans Python modules statically
• Extracts native docstrings during controlled build steps and persists them in exposure manifests
• Generates API references with stable import paths
• Builds documentation in an isolated environment

Documentation always reflects what is actually importable and executable.

No hand-written documentation layer exists alongside the generated output.

Runtime state: forge reconcile, forge deps, forge add-missing

The runtime itself - interpreters, uv, toolchains, and every resolved package - is governed state, not ambient machine configuration. It is pinned in a git-tracked, hash-locked manifest, and the engine keeps every machine honest against it.

• forge reconcile - the read-only verdict: re-hashes the live environment and compares it against the approved manifest. Runs on every activation; every run is retained in an append-only audit history. It never mutates anything - that is why its verdict is trustworthy.
• forge deps - the collaboration view: compares the live environment against what the current branch declares, and records a developer's own additions to a git-tracked branch record - automatically, on activation.
• forge add-missing - the catch-up verb: installs whatever the branch declares that the local environment is missing, including packages a teammate added - one command, no admin involvement.

Promotion of a branch's additions into the approved runtime is a separate, maintainer-only operation (forge admin promote): every promoted artifact is independently re-fetched and sha256-verified - never trusted from a developer's machine.

Developers remain free to experiment locally. The approved runtime changes only by deliberate, verified promotion - drift is measured, never guessed.

System diagnostics: forge doctor

The doctor command performs deep diagnostics of the project environment.

• Validates Python interpreter and virtual environments
• Detects import path inconsistencies
• Checks native artifact availability
• Identifies common misconfigurations

Doctor is designed for investigation and validation, not mutation.

Structural inspection: forge structure

The structure command reports the enforced project layout as a verified artifact.

It reflects the current, validated structure of the project - not a template or recommendation.

Cache hygiene

To keep execution deterministic across rebuilds and environments, FalconSmith treats Python cache artifacts as a source of nondeterminism.

Cache cleanup is handled explicitly to ensure that rebuilds, imports, and documentation generation reflect actual code state.

No engine operation relies on stale bytecode.

Why this workflow matters

Together, these operations enforce a system-wide invariant:

• The filesystem reflects intent
• The namespace reflects the filesystem
• Native artifacts reflect native sources
• Documentation reflects exposed behavior

Breakage in this chain is detected early and explicitly.

Relationship to verification

The engine workflow is inseparable from FalconSmith verification.

After each major operation, the system validates:

• Structural integrity
• Inventory consistency
• Interface visibility
• Documentation availability
• Test correctness

Verification is the mechanism that keeps the workflow trustworthy over time.

Summary

The FalconSmith engine is a reconciliation system, not a generator.

All workflow phases converge on a single deterministic system state defined by validated structure, exposure manifests, and inventory data.

Each engine command enforces correctness in one dimension of the project: namespace, native integration, structural exposure, and documentation.

Together, these components allow scientific Python libraries to evolve without accumulating structural drift, hidden dependencies, or undocumented behavior.