Query and datasets
Asking questions of the data once it is in.
Query, in the sidebar, is where you write SQL against your datasets. Every dataset in the tenant is
a table you can name in FROM, whatever brought it in: a sync, a webhook, a File Set, a pipeline,
or a registered or external dataset. Beetl runs the SQL on DataFusion with the PostgreSQL dialect,
and a query can read data but never change it.
Each run saves a result with its own URL. You can page through it, chart it, click a cell to see where a value came from, and download it as CSV. A result keeps at most 1,000 rows and expires after 7 days, so anything you need for longer, or in full, belongs in a dataset written by a pipeline. Query results covers the limits, ownership and retention.
In this section
Guides
| Page | What it covers |
|---|---|
| The SQL editor | Autocomplete, multi-tab, cancellation, and addressable results. |
| Result charts | Plotting a result as a bar, line, area, scatter or pie chart. |
| Lineage and drill-down | Following a cell back to the records behind it. |
| The Data Landscape graph | Every connection, dataset and pipeline on one map. |
| Export | CSV, and only CSV, up to 1,000 rows. |
SQL reference
| Page | What it covers |
|---|---|
| DataFusion SQL | The SQL Beetl runs: Apache DataFusion 54 with the PostgreSQL dialect, and where it differs from PostgreSQL. |
| JSON extraction | Reading fields, nested values and arrays out of JSON text, and turning them into typed columns. |
Related
Last updated on