Core concepts

Ten nouns. Get these right and most misunderstandings on a call never happen. Each one links to its own page where there is more to say.

Tenant and roles

A tenant is one customer workspace. Everything else lives inside exactly one of them, and no request carries a tenant id that Beetl trusts: scope is always derived from the caller's session or credential. There are two roles, TenantAdmin and TenantUser, and they differ in exactly one place. Registering, revalidating and switching an object store is TenantAdmin-only. Automations, pipelines, connections, datasets, API keys and MCP scopes are not gated by role at all. Tenants and roles.

Object store

Your S3-compatible bucket, registered by a TenantAdmin, validated with a live write, read and delete round-trip. Beetl provisions none for you. A new tenant has an empty store list, and until the first one is registered every write path returns OBJECT_STORE_NOT_CONFIGURED. The first store registered becomes the tenant default. Because the bucket is yours, this is also where the data-residency conversation ends. Tenants and roles.

Connection

A Connection is a configured link to one source system: a PostgreSQL database, a webhook endpoint, an HTTP feed, an uploaded file, a Business Central or SAP B1 instance, or a Server-Sent Events stream. It holds the credentials, the schedule, and the selection of which tables or entities to extract. It has a lifecycle you can see: Pending, Active, Paused, Failed, Deleted. Connections and Integrations.

Integration, and the word "connector"

Three words in this product have been used for overlapping things, so be precise. An Integration is the official term for a custom connector, meaning the code that knows how to talk to a particular system. A Connection is one configured instance of a link to a source system, as above. beetl-connector is the edge binary you run as a Docker container inside your own network so that Beetl can reach systems it cannot dial from outside. "Connector" was retired from the UI on 2026-08-11 in favour of "Integration", but it survives as the name of that binary, so read it as the edge agent whenever you see it. Connections and Integrations.

Dataset

A Dataset is a table Beetl owns. It records where it came from, either a data source or a pipeline, and it carries a full Arrow schema, partition columns, a version, and the object store it was written against. Datasets are created for you: one per uploaded file, one per webhook or feed channel, one per pipeline destination. Datasets and tiers.

Delta Lake and the tiers

Datasets are Delta Lake tables, which is where ACID writes, versioning and time travel come from, and Beetl runs compaction, vacuum and checkpointing on them automatically. Each one is labelled Bronze, Silver or Gold. Treat that label as a shared convention about how processed the data is, not as a constraint: there is no enforced tier lineage DAG, and nothing prevents a Gold table reading from another Gold table. Datasets and tiers.

Pipeline

A Pipeline is a set of source datasets, an ordered list of SQL stages, and an optional destination dataset. Every pipeline exists in two versions at once. Working is the draft you edit and preview. Deployed is the version that actually executes. Save writes the draft, Deploy promotes it, and the version badge tells you which state you are in. A pipeline with no destination is valid and is how exploration usually starts, though it cannot be run. Pipelines.

Run and job

A run is one execution of a deployed pipeline or an automation. Underneath, a job is the unit the platform schedules: pipeline execution, query execution, extraction, connectivity testing, file ingestion, or Delta maintenance. Jobs move Queued, Dispatched, Running, then Completed, Cancelled or Failed, and they record rows read and written, bytes, and duration. Visibility is uneven today. A pipeline's latest run shows on its graph node, automations keep full run history, and there is no general jobs page. Runs and jobs.

Query result

Every successful query produces a durable, addressable artifact at /web/query-results/{result_id}, stored as Arrow IPC in your object store with its metadata in Postgres. It is creator-owned and tenant-scoped, it can be paged, charted, drilled into and exported as CSV, and it expires seven days after it was created with no pin and no warning. Query results.

The agent trust boundary

Beetl reads your systems, never writes back to them, and publishes only where you explicitly told it to. That is the accurate short form, and it has exceptions worth knowing before a security review: the assistant edits the draft state of the page you are on, it can build and deploy pipelines subject to scope and confirmation, and a deployed pipeline writes its destination dataset. The analysis plane cannot mutate anything, because the query engine is constructed with DML disabled. The agent trust boundary.