Files

Upload files from a working copy, attach and embed them in items, and keep their bytes on the device.

A file is two things: a blob, its bytes by their hash, and a file item that names the blob. A working copy queues the upload and the item as separate writes, each with its own verdict, and keeps fetched bytes in a cache beside the store. Files explains blobs and file items, and Keep files on a device explains the cache.

Add a file

CallQueuesReturns
items.addFile(_ file:_ Attachment, tags:)The upload, a file item attached to nothing, then a write for each tag.Added: upload and item.
items.attach(to:file:_ Attachment)The upload, a file item, and an attached-to edge from the file to the item.Attached: upload, item, edge and embed.
items.embed(file:in:_ Attachment)What attach queues, and an edit adding the embed to the end of the body after a blank line.Embedded: attached, body and embed.
blobs.put(file:mimeType:)The upload alone. The bytes are copied beside the store.The uploadBlob write.

Attachment(mimeType:title:type:tier:) sets what the file item is made as. A field left unset takes the file's.

Embeds

An embed is ![[title]] in an item's body. The body reads it back as a references edge from the item to the file.

  • Attached.embed is the text, or nil where the file's title can't name it alone, such as a title another attachment has or one ending in a document extension like .txt.
  • embed(file:in:_:) bases its edit on the body and version the copy holds, so a body changed elsewhere first is answered by the type's merge policy: conflicted for a note.
  • The body is read and written in two core calls, so a second edit of the same body queued meanwhile is replaced. Make an item's body edits one after another.
  • It throws EmbedFailure where the file was attached and the body step failed. Its attached holds the queued writes and cause says why. Any other error means nothing was written.
  • items.embedText(of:in:) gives the embed text for an attachment. It throws notFound for an item or file the copy doesn't hold, and invalid for an item that isn't an attachment of it or a title that can't name it alone.

items.links(in:) gives each embed's BodyTarget: .item(id:), the file item holding the bytes, .pending until the server's lookup answers, .missing, .ambiguous or .refused(reason:).

Read a file's bytes

let text = try await copy.blobs.withFile(hash) { url in
    try String(contentsOf: url, encoding: .utf8)
}

blobs.withFile(_ hash:_ body:) calls body with the local URL of the bytes, fetching them first when the cache doesn't hold them.

  • The file stays in the cache until body returns or throws, whatever the budget. A later trim can take it, so copy the file out if it must last longer.
  • A reader, or a copy with no server, throws bytesAbsent instead of fetching.
  • blobs.isHeld(_:) says whether the cache holds a blob, with no request.

The file cache

The cache holds fetched files under a budget. Files attached to items the copy holds stay outside the budget, so they're there offline once fetched.

CallDoes
blobs.cache()The FileCache, read with no request. status().fileCache holds the same.
blobs.setBudget(_ FileBudget)Keeps the choice in the store, trims at once, and returns the bytes taken.
blobs.trim()Trims to the budget now, and returns the bytes taken.
blobs.removeFetchedFiles()Removes every file the device can fetch again, and leaves the budget as it was.
blobs.fetchKept()Fetches every attachment the cache doesn't hold whole, smallest first.

FileBudget is one of these:

  • .default is the larger of 2 GiB and a tenth of the volume, held to what leaves a twentieth of the volume free.
  • .bytes(bytes:) sets a number of bytes.
  • .keepAll keeps every file fetched.

setBudget, trim and removeFetchedFiles throw readingHandle for a reader. removeFetchedFiles keeps:

  • the bytes of attachments
  • queued uploads
  • files being read in withFile
  • partial fetches

Where the cache can't be read, it takes nothing and throws store.

Fetch attachments

Run fetchKept() after a hydration and after a catch-up.

  • It resumes a partial file.
  • It waits out a server that asks for a wait of a minute or less.
  • It stops at the first file the disk has no room for, when the server can't be reached, or when the server asks for a longer wait.
  • A file that fails otherwise is passed over.
  • Canceling the task throws canceled, and the files fetched stay.
  • It throws readingHandle for a reader or a folder's copy, and noServer for a copy with no server.

KeptFetch has fetched, fetchedBytes and the attachments after it.

What the cache reports

FileCache fieldHolds
settingThe FileBudget chosen.
budgetThe budget in force, in bytes, or nil under .keepAll.
heldThe bytes kept, or nil where the cache couldn't be read, with the reason in heldUnreadable.
volumeUnreadableThe default budget is 2 GiB because the disk couldn't be read.
backupMarkFailureWhy fetched files couldn't be kept out of backups.
attachmentsKeptAttachments, or nil during a hydration, for a folder's copy, or where the cache couldn't be read.

KeptAttachments counts held and heldBytes, partialBytes, notHeld and notHeldBytes. Bytes are counted once for files that share them. Its notHeldByReason gives, for each NotHeldReason, the attachments, their bytes and how many state no size (unstated):

NotHeldReasonMeans
.notFetchedNo fetch was asked for since the file became an attachment.
.noRoomThe volume had no room for it, or for one before it in the same fetch.
.unreachableThe server or its link couldn't be reached, or stopped part way.
.rateLimitedThe server asked for a longer wait than a fetch makes.
.absentThe server holds no bytes by that hash.
.failedThe fetch failed otherwise.
.notInCopyThe edge is held but its file item isn't.

The rules are in the contract's blobs and device chapters.

On this page