Deploy and trigger

A pipeline writes its destination only after you deploy it. For the Working and Deployed versions, see Pipelines.

Save, Deploy and Run

Deploy is unavailable while you have unsaved changes. Previewing a stage never writes.

What Deploy checks

Beetl refuses the deploy unless all of these hold for the saved Working version:

1. The pipeline has a destination, and it is not a registered or external dataset.
2. Every source with Trigger on update can supply every run.* value the pipeline uses. If one cannot, the error names it, for example Source `orders` cannot satisfy run context: missing required values: run.dt.
3. Every run.* reference has a value. Preview and Deploy ask for these in a Provide run values dialog, which remembers your entries until you close the editor.
4. A dry run of all stages with those values succeeds. The dry run writes nothing.

The first run then starts with the run values you entered. If Deploy fails with a general error, the dry run failed: preview the last stage with the same run values to see the full message.

Three cases behave differently:

• If the Working version is identical to the Deployed version, Deploy changes nothing and starts no run. Use Run instead.
• If the pipeline is disabled, Deploy promotes the new version but starts no run.
• The dry run checks that your SQL runs, not that its result fits an existing destination. A changed column list or type fails the first run without writing. See Schema compatibility.

Run a deployed pipeline

Switch the editor header to Deployed and press Run, or choose Run pipeline on the pipeline's node on Home. If the Deployed version uses run.* values, a dialog asks for them.

Run stays unavailable until these are true, and its tooltip names the one that is missing:

Trigger on update

While a source card's Trigger on update switch is on, every write to that source by a sync, webhook, file processor or another pipeline starts a run. Registered and external datasets never trigger runs.

The run mapping

The run mapping decides which run.* values a triggered run receives. For a partitioned source, the card lists each partition column as a Source event field, such as {{ source.partition.values.dt }}, with an Available as name that starts as the column name. Rename it when your SQL uses a different name, for example day for a dt partition.

A write that touches several partitions starts one run per partition, each with that partition's values. A write to an unpartitioned source starts one run with no run.* values. When several sources trigger, each starts runs on its own; Beetl does not wait for the others to catch up.

Chat and MCP clients can also set mapping templates, such as {"dt": "{{ source.partition.values.invoice_date }}"}.

The WARN badge

A source card shows WARN when Trigger on update is on and the source cannot supply a run.* value the pipeline uses; hover to see which. Deploy refuses until you rename the Available as entry, change the template, or turn off Trigger on update for that source.

Runs wait their turn

A pipeline executes one run at a time, and the rest wait as Queued. If a source update arrives while a run with the same run values is still queued, Beetl does not queue another. Different pipelines run in parallel.

Enable and disable

Disable and Enable are on the pipeline's node on Home. A disabled pipeline starts no runs, whether from source updates, Run or a deploy, though you can still edit and deploy it. Updates it missed are not replayed when you enable it again, so run it by hand.

Read the Runs tab

Select the pipeline's node on Home and open Runs. Each row shows start time, status and duration; expand it for the error, the trigger, and the queued, started and finished times. A run moves through Queued, Dispatched and Running to Completed or Failed.

Deploy from chat or MCP

In chat, the assistant asks you to confirm before it deploys; save or dismiss unsaved editor changes first. If the pipeline changes before you confirm, the deploy is refused.

Over MCP, deploy_pipeline with a write key deploys at once, and the first run writes the destination. Pass run values as initial_run_context, for example {"run": {"dt": "2026-10-03"}}. Treat a write key like a deploy credential; see The agent trust boundary.

Neither chat nor MCP can rerun a deployed pipeline. run_pipeline only previews the stored Working version. To rerun, use Run in the app.

Run on a timetable

Pipelines have no schedule of their own. Trigger them from a sync that has one:

SELECT order_id, customer_id, amount, dt
FROM orders
WHERE dt = DATE '{{ run.dt }}'

Each sync run now replaces one day of output.

Related