Keep files on a device

A working copy holds items, not their bytes. It fetches a file's bytes when something reads them, keeps them in a file cache beside the store, and keeps the files attached to items it holds so they're there offline.

The CLI examples use marfa device, with the copy named by --db or MARFA_DB, as Write in a working copy shows. In the Swift examples, copy is an open working copy. Handle files covers uploading bytes and pointing an item at them.

Read a file's bytesCopy anchor link

Ask for the bytes by the hash in the file item's blob_ref. The copy fetches them first if it doesn't hold them:

marfa device blobs get sha256:abf95d4140df7b413a5ceb13f473a7b9e2ffbedac7854c4b412258be66c0da34

The CLI prints where the file is held. withFile calls your closure with it, and the file stays put only while the closure runs: read it inside the closure, and copy it out if it must outlast the call.

A reader, or a copy without a server, can't fetch, so it refuses a file it doesn't hold with bytes_absent.

Choose how much disk files takeCopy anchor link

The copy keeps the files it fetched within a file budget. Past it, a trim removes the files read longest ago, and a later read fetches them again. The default is the larger of 2 GiB and a tenth of the volume, held to what leaves a twentieth of the volume free.

marfa device blobs budget 5000000000

Setting a budget trims to it at once and reports the bytes removed, and the store keeps the choice. The CLI also takes default and keep-all, and the Swift SDK .default and .keepAll. A file larger than the budget is still fetched when asked for.

To empty the file cache without changing the budget, such as for a "Remove downloaded files" button, a Swift app calls copy.blobs.removeFetchedFiles(). It removes every file the copy can fetch again, reports the bytes removed and leaves the person's budget as it was. Attachments kept offline, uploads not yet sent and files being read stay. The CLI has no command for it.

Keep attachments offlineCopy anchor link

A file attached to an item the copy holds stays on the device, outside the budget, and a trim never takes it. Fetch every one the copy lacks after a hydration and after each catch-up:

marfa device blobs fetch-kept

It fetches the smallest files first and waits out a server that asks for a wait of a minute or less. It stops at the first file the disk has no room for, once the server can't be reached, or once the server asks for a longer wait, and passes over a file that fails otherwise. Ctrl-C, or canceling the task, stops it, and the files already fetched stay.

When a fetch stops part wayCopy anchor link

When the link breaks or goes quiet, the copy keeps the bytes it received as a partial file, though it never answers them as the file. The next fetch of those bytes asks the server for a new link and requests the rest with a Range header. When the link sends the whole file instead, the copy starts again from zero.

A partial file that isn't an attachment's counts toward the budget, and a trim can take it. Each time the copy is opened, other than as a reader, it also removes partial files left untouched for seven days, the longest a link lasts, again unless they're an attachment's.

When a fetch is refusedCopy anchor link

A fetch that can't keep the file whole refuses with one of these codes:

The Swift SDK throws these as noRoom, storageFull, bytesAbsent and decoding.

Find out why an attachment is missingCopy anchor link

marfa device status counts the attachments the copy holds whole, with the bytes they and their partial files take, and the attachments it lacks, grouped by why. Under --json the groups are file_cache.attachments.not_held_by_reason, each with a reason, the count of attachments, the bytes they state and how many are unstated:

The Swift SDK reads the same counts from blobs.cache(). On Apple platforms, the copy also keeps fetched files out of backups.

The rules behind this guide are in the device chapter of the contract.