TamarackDB

Configuration

Every setting of tamarackdb-server. The settings of tamarackdb-backup are in Backup.

Key words in capitals follow RFC 2119.

Sources

A setting comes from, in this order:

  1. A TOML file, passed with --config (config.toml in the working directory by default). Settings live under a [server] section.
  2. A TAMARACKDB_* environment variable, one per setting.
  3. A built-in default, for the settings that have one.
  • A value set in the file always wins over the environment variable.
  • The file is optional. Use one file per instance in production. In Docker, environment variables cover a deployment with no file at all.
  • The file is checked as a whole, [server] and [backup] sections included. An unknown key, or a key outside any section, stops the server at startup with an error naming it, so a typo never leaves a setting at its default.
  • The same file can hold the [backup] section of tamarackdb-backup. Each binary reads only its own section.

Generate a starter file, with every setting commented out at its default:

./bin/tamarackdb-server --default-config > config.toml
./bin/tamarackdb-backup --default-config >> config.toml

A file that holds authToken is a secret: see Security.

Settings

KeyEnvironment variableDefaultWhat it sets
socketPathTAMARACKDB_SOCKET_PATH/run/tamarackdb/tamarackdb.sockThe unix socket to listen on. At most 107 bytes, the Linux limit
socketModeTAMARACKDB_SOCKET_MODE"0600"The socket’s permissions, as an octal string. Only with socketPath
bindAddressTAMARACKDB_BIND_ADDRESS127.0.0.1The address to listen on over TCP
portTAMARACKDB_PORT8085The port to listen on over TCP
enableAuthTAMARACKDB_ENABLE_AUTHfalseWhether every request needs the Bearer token
authTokenTAMARACKDB_AUTH_TOKENnoneThe Bearer token
dataDirTAMARACKDB_DATA_DIRdataThe directory holding the database file
logLevelTAMARACKDB_LOG_LEVELwarningThe lowest level logged: debug, info, warning, or error (see Logs)
devModeTAMARACKDB_DEV_MODEfalseTurns on POST /reset and the profiling endpoints (see Development mode)
defaultEventsPerPageTAMARACKDB_DEFAULT_EVENTS_PER_PAGE1000The limit of a QUERY /events that leaves it out
maxEventsPerPageTAMARACKDB_MAX_EVENTS_PER_PAGE10000The highest limit a QUERY /events may ask for
maxEventSizeTAMARACKDB_MAX_EVENT_SIZE65536 (64 KiB)The largest event, in bytes (see Events)
maxProjectionSizeTAMARACKDB_MAX_PROJECTION_SIZE65536 (64 KiB)The largest projection, in bytes: its type, id, and payload together
maxEventsPerWriteTAMARACKDB_MAX_EVENTS_PER_WRITE100The most events, and the most Append Conditions, in one POST /write
maxProjectionsPerWriteTAMARACKDB_MAX_PROJECTIONS_PER_WRITE500The most projections in one POST /write, across its three lists
maxRequestBodySizeTAMARACKDB_MAX_REQUEST_BODY_SIZE8388608 (8 MiB)The largest request body, in bytes, for every endpoint
maxQueuedWritesTAMARACKDB_MAX_QUEUED_WRITES100The most requests waiting for their turn at once (see below)
readPoolSizeTAMARACKDB_READ_POOL_SIZE8SQLite connections for reads, and so how many reads run at once

Listening

  • By default, the server listens on the unix socket at socketPath. Setting bindAddress or port switches it to TCP. socketPath wins whenever it’s set, even alongside them.
  • The default socket’s directory, /run/tamarackdb, MUST exist and be writable by the server’s user. Under systemd, RuntimeDirectory=tamarackdb creates it (see Install); elsewhere, create it yourself, or set a path the server’s user owns.
  • At startup, the server removes a socket left at that path by an earlier run, and refuses to start if the path holds anything other than a socket.
  • The server speaks plain HTTP either way. Why the socket is the recommended setup, and how to reach the server from another host, is in Security.

Data directory

  • dataDir holds the database file, tamarackdb.sqlite. Only the directory is configurable: the file name is fixed.
  • Its layout is managed by TamarackDB and may change between versions: don’t rely on it, and don’t edit its contents by hand.
  • Its permissions are in Security.

Limits

  • The size and count limits (maxEventSize, maxProjectionSize, maxEventsPerWrite, maxProjectionsPerWrite, maxRequestBodySize) are a cautious starting point. Find the real limits in development, with the application’s data, then set the same values in production. Every error from a limit names the setting to raise (see Writing).
  • maxRequestBodySize isn’t checked against the other limits: it’s the real bound on a write, and the others are rules for each item.
  • defaultEventsPerPage and maxEventsPerPage are settings, not constants, because how fast a projector processes a batch varies between applications, and between projectors of one application. The defaults keep a default page easy to buffer, and a maximum page done in seconds.
  • Every limit MUST be positive, and defaultEventsPerPage MUST NOT exceed maxEventsPerPage: the server refuses to start otherwise.

Sizing the write queue

maxQueuedWrites bounds how many requests wait for their turn at once: POST /write, the bulk deletes of projections, POST /reset, and the hourly PRAGMA optimize (see The write FIFO). One more gets 503 WriteQueueFull instead of joining.

  • A write holds the turn only while its own SQLite transaction runs, usually a few milliseconds, so the queue is usually empty or short. Reads never wait in it.
  • A large write holds the turn longer: a projection rebuild sent in one write holds it for as long as its inserts take, and the writes behind it wait that long.
  • Size maxQueuedWrites against how many writes the application sends at once. There’s no “no limit” value: every deployment gets a bound.
  • The server doesn’t cap how long a request waits. Each client sets its own limit and closes the connection when it’s reached.
Prev
Install
Next
Security