Export and restore

Restore from an archive

Restores an exported archive atomically and returns counts of written and skipped records. While it writes, Marfa turns other writes away. Requires backups.manage, config.manage to add settings, and a sign-in from the last fifteen minutes.

POST
/restore

Authorization

bearerAuth
headerAuthorizationBearer <token>

Authenticate with an API key (marfa_k1_…) or an OAuth access token (marfa_at_…).

Header parameters

X-Marfa-Instance-Id?string

The ID of the instance you expect, as GET / returns it. If another instance answers at this address, Marfa returns 409 instance_changed and doesn't write. Leave it out to send the request to whichever instance answers.

Match^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

Request body

The archive file, as GET /export?format=archive returned it, with its manifest.json, files.ndjson.gz, instance.ndjson.gz, items.ndjson.gz, versions.ndjson.gz, edges.ndjson.gz and rows.ndjson.gz, in that order. Only the build that wrote an archive is sure to read it.

body*fileapplication/x-tar
Formatbinary

Response body

About errors

Returns the counts. Items and edges keep their IDs, versions, dates and history, even if their type now refuses them or is gone. An item whose ID, natural key or link is taken is left as it is, and a setting, icon, source name or connector state is set only where none is. An archive holds no sign-in, keys, apps, webhooks or tombstones.

application/json
  1. response
imported*number

How many items Marfa wrote.

duplicates*number

How many items Marfa skipped because their ID, natural key or link is already taken.

edges_imported*number

How many edges Marfa wrote.

edges_skipped*number

How many edges Marfa skipped.

edges_skipped_reasons*

How many edges Marfa skipped for each reason, such as endpoint_missing or already_present.

blobs_imported*number

How many blobs the archive carried whose bytes match their hash, whether or not the instance already had them.

entries_skipped*array<string>

The name of each archive entry Marfa doesn't know and skipped, such as one a tar tool adds, in the order the archive holds them.

line_kinds_skipped*array<string>

Each kind of instance.ndjson.gz line Marfa doesn't know and skipped, such as one a later build of the same archive version adds, named once in the order the archive first holds it.

blobs_missing*array<string>

The hashes of the blobs the archive names but doesn't carry with matching bytes, and that Marfa holds no copy of. Upload their bytes again with POST /blobs to restore them.

types_registered*number

How many types Marfa registered.

types_skipped*number

How many types Marfa skipped because the instance already registers them identically.

edge_types_registered*number

How many edge types Marfa registered.

edge_types_skipped*number

How many edge types Marfa skipped because the instance already registers them identically.

type_icons_added*number

How many of the owner's type icons Marfa added, for types the instance had no owner icon for.

type_icons_skipped*number

How many of the owner's type icons Marfa skipped.

type_icons_skipped_reasons*

How many of the owner's type icons Marfa skipped for each reason: already_set when the instance already has an owner icon for the type, unknown_icon when the icon isn't the ID of an icon in a set Marfa pins.

sources_added*number

How many source names and marks Marfa added, for sources the instance names nothing yet.

sources_skipped*number

How many source names and marks Marfa skipped.

sources_skipped_reasons*

How many source names and marks Marfa skipped for each reason: already_set when the instance already has a name or mark for the source, unknown_mark when the mark isn't the ID of an icon in a set Marfa pins, duplicate_code when another line of the archive lands on the same code and the line already carrying it wins.

settings_added*number

How many settings Marfa stored, for settings the instance had not stored.

settings_skipped*number

How many settings Marfa skipped because the instance already stores them.

settings_dropped*array<string>

The names of the settings the archive carried that this instance does not have. Marfa left them out and stored nothing for them.

settings_without_receiver*array<string>

The reporting switches Marfa stored as on that send nothing, because no receiver is set: heartbeat_enabled and error_reporting_enabled.

enforcement_added*array<>

The archive's enforcement settings Marfa stored, each one the instance didn't store.

enforcement_skipped*array<>

The archive's enforcement settings Marfa skipped, because the instance already stores each one or stores enforcement settings that don't read.

connector_states_added*number

How many connector state documents Marfa stored, for sources the instance held none for.

connector_states_skipped*number

How many connector state documents Marfa skipped because the instance already holds one for the source.

connector_agreements_added*number

How many connector agreements Marfa stored.

connector_agreements_skipped*number

How many connector agreements Marfa skipped because the instance holds no such item or already holds an agreement for it.

rows_imported*number

How many table rows Marfa wrote.

rows_skipped*number

How many table rows Marfa skipped because their item was counted as a duplicate.

not_restored*array<>

Each kind of thing an archive never holds, so the owner sets it up again on this instance: the owner's sign-in, keys, apps, webhooks, inbound endpoints, connector keys, the instance's addresses and name, and the logs. Then enforcement when the instance that wrote the archive couldn't read its enforcement settings, so the archive doesn't hold them.

previous_instance?

The instance that wrote an archive, and what it keeps in this instance's backup destination. Present when the archive was written by another instance than this one.

secrets_to_set*array<string>

The secret settings the archive's instance had set that this instance holds no value it can read for. Set each again; the attention list names them until you do.

  • validation_error: the body isn't an archive, lacks or repeats an entry, is from before 0.0.72 or a later version, or has an invalid line or one over 64 MiB.
  • invalid_properties: a core or system item breaks its type, or strict_mode refuses a property.
  • invalid_schema, inheritance_violation, property_shadows_field: an archived type is refused as POST /types would refuse it.
application/json
  1. response

An error response.

error*

What went wrong.

  • unauthorized: the request has no credential, or its credential is not valid. For a key that has expired, details.expired_at says when it expired.
  • insufficient_user_authentication: your app's owner signed in more than fifteen minutes ago. Send them through GET /oauth/authorize with max_age.
application/json
  1. response

An error response.

error*

What went wrong.

  • forbidden: you don't hold backups.manage, or the archive would store settings and you don't hold config.manage. details.required_scope names the missing permission. For config.manage, details.settings lists the settings.
  • app_sign_in_required: you are an API key. A person restores from an app they signed in to, or with the private local command.
application/json
  1. response

An error response.

error*

What went wrong.

  • request_timeout: no bytes of the archive arrived for 60 seconds. Nothing was written. An upload has no other time limit while bytes keep arriving.
application/json
  1. response

An error response.

error*

What went wrong.

  • conflict: the archive redefines a type this instance registers differently or registered while the restore ran, carries a core edge type, or carries a snapshot ID that already exists.
  • link_taken: the archive registers a type naming a link_field in which two items a forced delete left share a value.
application/json
  1. response

An error response.

error*

What went wrong.

contract_mismatch: X-Marfa-Contract names another contract version than this instance speaks, or is not a whole number. Nothing was done.

application/json
  1. response

An error response.

error*

What went wrong.

  • request_too_large: an item's properties, the properties of one of its earlier versions, or an edge's properties are larger than the bulk write endpoints accept. details names the line and the field.
application/json
  1. response

An error response.

error*

What went wrong.

compatible_with_violation: an archived type's compatible_with names a type neither the archive nor this instance holds, or one the type doesn't satisfy.

application/json
  1. response

An error response.

error*

What went wrong.

rate_limited: you sent too many requests. Wait for the number of seconds in Retry-After, then try again.

application/json
  1. response

An error response.

error*

What went wrong.

internal_error: Marfa failed in a way it didn't expect, and the request may not have completed. Read what you changed before you repeat a write.

application/json
  1. response

An error response.

error*

What went wrong.

  • blob_store_unavailable: a blob store failed, or didn't answer in time, while Marfa checked a blob's copy, or a store hasn't finished an earlier write or delete of a blob the archive names. Nothing was written. Try again later.
  • write_contention: the database was busy, and Marfa couldn't complete the request in time. Nothing changed. Try the request again.
application/json
  1. response

An error response.

error*

What went wrong.

insufficient_storage: the disk that holds the instance's data has no room for the request, or the request would leave less free than the instance keeps in reserve. Nothing changed, unless details.write_outcome is unknown, which means the write may have landed: read what you changed before you repeat it. Free space on the disk, then try the request again.

application/json
  1. response

An error response.

error*

What went wrong.

curl -X POST 'http://localhost:8600/restore'
About errors
{  "imported": 0,  "duplicates": 9,  "edges_imported": 0,  "edges_skipped": 0,  "edges_skipped_reasons": {},  "blobs_imported": 0,  "blobs_missing": [],  "types_registered": 0,  "types_skipped": 3,  "edge_types_registered": 0,  "edge_types_skipped": 0,  "type_icons_added": 0,  "type_icons_skipped": 0,  "type_icons_skipped_reasons": {},  "sources_added": 0,  "sources_skipped": 1,  "sources_skipped_reasons": {    "already_set": 1  },  "settings_added": 0,  "settings_skipped": 0,  "settings_dropped": [],  "settings_without_receiver": [],  "enforcement_added": [],  "enforcement_skipped": [],  "connector_states_added": 0,  "connector_states_skipped": 0,  "connector_agreements_added": 0,  "connector_agreements_skipped": 0,  "rows_imported": 0,  "rows_skipped": 0,  "entries_skipped": [],  "line_kinds_skipped": [],  "not_restored": [    {      "kind": "sign_in",      "message": "The owner's sign-in is not restored. Sign in again on this instance."    },    {      "kind": "keys",      "message": "Keys are not restored. Make new keys on this instance."    }  ],  "previous_instance": {    "instance_id": "01a10128-f3e1-7ab1-b58c-a89836a748a8",    "destination": "folder",    "kept_by_previous_instance": [      "backups/01a10128-f3e1-7ab1-b58c-a89836a748a8",      "backup-parts/01a10128-f3e1-7ab1-b58c-a89836a748a8",      "files/01a10128-f3e1-7ab1-b58c-a89836a748a8"    ],    "message": "Instance 01a10128-f3e1-7ab1-b58c-a89836a748a8 wrote this archive. If it took backups to this backup folder, its backups, their parts and its file copies are in backups/01a10128-f3e1-7ab1-b58c-a89836a748a8, backup-parts/01a10128-f3e1-7ab1-b58c-a89836a748a8 and files/01a10128-f3e1-7ab1-b58c-a89836a748a8. Delete them only once that instance no longer runs and nothing else restores from them."  },  "secrets_to_set": []}