Pull from a vendor

A connector pulls: it reads the vendor and writes what it finds as items. A two-way connector also carries changes made in Marfa back, as Carry changes back shows, and a one-way connector says which types it holds read only.

The operations below come in the order a run uses them, for a connector built directly on the API. The connector kit does all of this for you, read-only types included.

The examples follow a task manager connector with a key of its own under the source tasks, which writes the type tasks.task. In the TypeScript examples, marfa is a TypeScript SDK client made with that key.

Write what you readCopy anchor link

A pull reads the vendor from where the last run stopped, which the connector keeps in its state document, and writes what it read. Upsert items in bulk writes up to 5,000 records in one request:

POST /items/bulk
Authorization: Bearer <key>
Content-Type: application/json

{
  "items": [
    {
      "type": "tasks.task",
      "source_id": "acct-7:4012",
      "tier": "feed",
      "properties": { "task_id": "4012", "title": "Renew passport", "status": "not_started" },
      "occurred_at": "2026-10-01T08:00:00Z"
    }
  ],
  "atomic": false
}

Marfa answers each record on its own:

{
  "counts": { "created": 1, "updated": 0, "skipped": 0, "errored": 0 },
  "results": [{ "index": 0, "outcome": "created", "id": "01a12313-fd4a-78a1-af1e-32792c3df63b" }]
}

An item's natural key is its source, from the connector's key, and its source_id, the vendor's own ID for the record. Marfa updates the item that holds the key, or creates one, so reading a record twice never makes a second item. Put the vendor account inside the source_id, as in acct-7:4012, so two accounts under one source can't share an item.

tier: "feed" puts the item in the feed, where a person or an app decides whether to keep it. occurred_at is when the record happened at the vendor. atomic: false writes the records that pass and reports a refused one as errored; by default, one refused record rolls the whole batch back. Marfa answers reads while a page is written. Under the default, a read shows each item as it was before the page until the page ends, and other writes wait for the page, so a large atomic page holds up every other writer.

Write an item's text as Markdown. Marfa stores the field its type names in display_hints.body_field unchecked, so convert a vendor's HTML first, or an app shows the tags.

Write only what changedCopy anchor link

A write of the properties an item already holds still raises its version and announces item.updated, so a pull that rewrites every record makes noise for everyone who follows the event stream.

Before you write, look up the items you hold under the records' keys and send only the records that differ:

POST /items/lookup
Authorization: Bearer <key>
Content-Type: application/json

{ "type": "tasks.task", "source": "tasks", "source_ids": ["acct-7:4012", "acct-7:4013"] }

Two cases need care:

• An item in the bin: a record that matches it comes back skipped, with the reason trashed.
• A purged item: it leaves a tombstone, which the lookup returns beside the items. Write a purged record again only if the vendor changed it after the tombstone's settled_at.

Keep a quiet run lightCopy anchor link

A run that finds nothing new at the vendor should send Marfa a few requests, however much the connector holds. Ask about many records at once: one lookup names many source_ids, and a listing reads up to 500 items a page. Never send a request for each item you hold.

In the kit, linkedEach asks about many targets in one call, so use it rather than linked in a loop, and call held once for each type in a run. Every connector in marfa-connectors keeps the kit's shared quiet-run test, which fails when a run with nothing new sends Marfa more than one more request for each 200 rows Marfa holds.

Set aside what the vendor deletedCopy anchor link

When the vendor no longer has a record, change the item's state to archived. Don't move it to the bin: the bin is for people, and an archived item keeps its history and can be made active again if the record comes back.

POST /items/01a12313-fd4e-7800-ab67-c886f1b9ef58/transition
Authorization: Bearer <key>
Content-Type: application/json

{ "state": "archived" }

Be sure the vendor deleted it: a partial listing, such as open tasks only, says nothing about a record it leaves out, so ask about that record first.

Declare the types you don't carry backCopy anchor link

A connector that takes no edits back for a type puts back, on its next run, any edit a person makes to the type's items in Marfa. Name those types in read_only_types when you register the connector, and their items read read_only: true, so an app can open them locked:

POST /connectors
Authorization: Bearer <key>
Content-Type: application/json

{ "name": "tasks", "read_only_types": ["tasks.project"] }

• Each registration replaces the list, and one without it clears the list. Build it on every start from what the connector carries back with the settings it runs on. The kit does this for you: it lists every type the connector writes and doesn't carry back, so a settings change reaches the list when the connector restarts.
• An item reads true only while every connector whose live key owns or claims its source lists its type and may write it. Once that stops, its next read says false.
• Marfa doesn't refuse an edit to a read-only item. The flag says what the connector will do.
• The list is per type. Leave a type out when you carry back any of its fields or items.

The rules behind this page are in the connectors and items chapters of the contract.