SDK reference

@beetlio/connect is the TypeScript SDK you write Connect packages with. The SDK is open source under the Apache-2.0 license: the full source and type definitions are on GitHub, and its user guide is at beetlio.github.io/connect. This page covers version 0.6, which needs Node.js 24.2 or newer.

npm install --save-dev @beetlio/connect

Entry points

Exports

Types such as SyncContext, Batch and RetryPolicy are exported too.

The SDK can also define destinations, which write records to an API. Beetl runs a package's syncs, and a package installed in Beetl must declare at least one sync.

The integration definition

import { auth, defineIntegration, secret, z } from "@beetlio/connect";

const Order = z.object({ id: z.string(), total: z.number(), status: z.string() });
const Page = z.object({ orders: z.array(Order), next: z.string().nullable() });

export default defineIntegration({
  key: "acme",
  displayName: "Acme",
  connection: {
    origin: { type: "input", input: "baseUrl" },
    inputs: z.strictObject({ baseUrl: z.url().meta({ title: "API URL" }) }),
    auth: auth.custom({
      credentials: z.strictObject({ apiKey: secret(z.string().min(1)) }),
      headers: { "x-api-key": { credential: "apiKey" } },
    }),
    async verify(ctx) {
      await ctx.json("/me", z.object({ id: z.string() }));
    },
  },
  syncs: (sync) => ({
    orders: sync({
      records: Order,
      inputs: z.strictObject({ status: z.enum(["open", "closed"]).default("open") }),
      async *run(ctx) {
        for await (const page of ctx.paginate({
          request: {
            path: "/orders",
            query: { status: ctx.config.sync.status },
          },
          schema: Page,
          next: ({ data, request }) =>
            data.next === null
              ? undefined
              : { ...request, query: { ...request.query, cursor: data.next } },
        })) {
          yield { records: page.data.orders };
        }
      },
    }),
  }),
});

This example reads every order with the chosen status on each run. Its pagination cursor is used only within that run, so it suits the Replace write mode, which the person who adds the sync chooses. For incremental reads with a checkpoint that carries across runs, see Authoring a sync.

defineIntegration

connection

Input schemas become forms, so they cannot use transforms or custom refinements. Set labels and help text with .meta({ title, description }).

sync(definition)

Batches

Each value run yields is one batch. Beetl accepts a batch's records and checkpoint together before the generator continues; the dataset changes only when the run succeeds.

A sync does not choose a write mode, declare a primary key or report deletions. The person who adds the sync chooses Append or Replace. See Write modes.

A batch holds at most 10,000 records. For the per-run limits, see Write modes.

Sync context

run(ctx) receives a SyncContext.

ctx.paginate takes a request (path, optional query and headers), a schema for each page, and a next function. next receives the page and returns the next request, or undefined to stop. Each page has data, response (status and headers), request and index. A pagination stops with an error after 10,000 pages or on a repeated request.

Await every ctx call. The SDK adds credentials to each request, so sync code never handles secrets.

Auth helpers

Custom and token-exchange credentials are strict objects of strings, and they cannot have defaults. Wrap sensitive fields in secret(). See Authentication.

Helpers

batchRecords(source) takes an iterable or async iterable of records and yields record-only batches of a suitable size. Use it as yield* batchRecords(rows) when a sync has no checkpoint; yield batches yourself when each one carries a checkpoint.

storageRecord(schema) wraps a provider object schema so every top-level field gets a fixed column type. Strings, numbers, booleans, arrays and strict nested objects keep their type. A union of scalar types is stored as text, and other dynamic values, such as records with arbitrary keys, are stored as JSON text. The schema must be a plain object without a catchall or root refinements.

import { auth, batchRecords, defineIntegration, storageRecord, z } from "@beetlio/connect";

const Customer = z.object({
  id: z.number(),
  name: z.string(),
  attributes: z.record(z.string(), z.unknown()),
});

export default defineIntegration({
  key: "acme-customers",
  displayName: "Acme customers",
  connection: { origin: "https://api.example.com", auth: auth.bearer() },
  syncs: (sync) => ({
    customers: sync({
      records: storageRecord(Customer),
      async *run(ctx) {
        yield* batchRecords(await ctx.json("/customers", z.array(Customer)));
      },
    }),
  }),
});

CLI

The SDK installs the beetl-connect command. Run it with npx beetl-connect, and add --help to any command for every option.

The sync argument is optional when the integration has one sync. Local credentials are saved unencrypted in files only your user can read, in the operating system's user configuration directory. See Local development and Pack and upload.