Configuration Reference

This document describes the TOML configuration file format used by Language Pipes (LpConfig). See the CLI Reference for how -c/--config resolves a configuration file and how it is used by run, config, and the TUI.

A small set of machine-local settings are controlled by environment variables instead of the TOML file. See Environment Variables below.

---

Minimal Configuration

node_id = "my-node"
job_port = 8000
end_models = ["Qwen/Qwen3-1.7B"]

[[layer_models]]
model_id = "Qwen/Qwen3-1.7B"
device = "cpu"
memory = 4

Key ordering matters. In TOML, key/value pairs that appear after an array-of-tables header ([[layer_models]], [[bootstrap_nodes]], [[end_models]]) belong to that table, not to the top-level document. Put all top-level scalar keys (node_id, job_port, string-form end_models, etc.) before any array-of-tables blocks. When using the table form of end_models, place those [[end_models]] blocks alongside your other array-of-tables sections.

Complete Example

# === Required ===
node_id = "node-1"

# === End Models ===
end_models = ["meta-llama/Llama-3.2-1B-Instruct"]

# === API Server ===
job_port = 8000
api_keys = ["test_key"]

# === Network ===
peer_port = 5000
network_ip = "192.168.0.1"
network_key = "9f86d081884c7d659a2feaa0c55ad015"
whitelist_node_ids = []

# === Layer Models ===
[[layer_models]]
model_id = "meta-llama/Llama-3.2-1B-Instruct"
device = "cpu"
memory = 5

[[layer_models]]
model_id = "Qwen/Qwen3-1.7B"
device = "cuda:0"
memory = 8

# === Bootstrap Nodes ===
[[bootstrap_nodes]]
address = "192.168.0.2"
port = 5000

---

Properties

Required

node_id

Unique identifier for this node on the network.

Type	Required	Default
string	✓	—

node_id = "my-node-1"

layer_models

Array of models to host. Each model is defined as a TOML table.

Type	Default
array of tables	[] (empty)

[[layer_models]]
model_id = "Qwen/Qwen3-1.7B"
device = "cpu"
memory = 4
data_type = 16

Each entry has the following fields:

Field	Type	Required	Description
model_id	string	✓	HuggingFace model ID or path in /models directory
device	string	✓	PyTorch device: cpu, cuda:0, cuda:1, etc.
memory	number	✓	Maximum memory allocation in GB
data_type	string	x	Set to 16, 8, or 4 to set quantization level

Note: Setting the data_type property to 8 or 4 requires the bitsandbytes library to be installed. Install it with pip install language-pipes[quantization] or pip install bitsandbytes.

Multiple models:

[[layer_models]]
model_id = "Qwen/Qwen3-1.7B"
device = "cpu"
memory = 4

[[layer_models]]
model_id = "meta-llama/Llama-3.2-1B-Instruct"
device = "cuda:0"
memory = 8

end_models

Array of end models to load (embedding layer + output head). The node with a model in its end_models list is the only node that can see your actual prompts and responses for that model. Other nodes only process hidden state tensors and cannot read the conversation content.

Type	Default
array of strings or tables	[] (empty)

Each entry is either a plain model ID string, or a table with a model_id and options:

Field	Type	Required	Default	Description
model_id	string	✓	—	HuggingFace model ID or path in /models directory
num_local_layers	int		1	Number of initial model layers the end model executes locally before forwarding work to other nodes. Higher values improve prompt obfuscation by keeping more of the early pipeline on your machine. All nodes hosting the same end model should use the same value so that model layers are loaded correctly.
device	string		cpu	PyTorch device (cpu, cuda:0, cuda:1, …) used for both the local layers and the embedding/output head modules of this end model.

Simple form (one local CPU layer each):

end_models = ["Qwen/Qwen3-1.7B"]

Table form when you need per-model options:

[[end_models]]
model_id = "Qwen/Qwen3-1.7B"
num_local_layers = 2
device = "cuda:0"

Because TOML arrays cannot mix strings and tables, use one form for the whole end_models list. Any model that doesn’t set num_local_layers defaults to 1, and any that doesn’t set device defaults to cpu.

Local end model setup:

end_models = ["Qwen/Qwen3-1.7B"]

[[layer_models]]
model_id = "Qwen/Qwen3-1.7B"
device = "cpu"
memory = 2

---

API Server

job_port

Port for the OpenAI-compatible API. Omit to disable the API server.

Type	Default
int	None (disabled)

job_port = 8000

api_keys

List of accepted API keys for the OpenAI-compatible server.

Type	Default
array of strings	None (disabled)

api_keys = ["test_key"]

max_node_jobs

Maximum number of jobs this node will queue for a single peer node. Incoming jobs from a node whose queue is already full are rejected. Configurable from the TUI’s “Jobs / Server” page.

Type	Default
int	10

max_node_jobs = 10

max_api_jobs

Maximum number of pending jobs allowed per API key on the OpenAI-compatible API. Requests beyond this limit are rejected until earlier jobs for that key complete. Configurable from the TUI’s “Jobs / Server” page.

Type	Default
int	5

max_api_jobs = 5

---

Network

These options configure the peer-to-peer network. See Distributed State Network for details.

peer_port

Port for peer-to-peer communication.

Type	Default
int	5000

peer_port = 5000

network_ip

IP address that this node advertises to other peers. Only necessary for bootstrap configurations where other nodes connect to this node. If not specified, the node attempts to auto-detect its network IP.

Type	Default
string	None (auto-detect)

network_ip = "192.168.1.100"

bootstrap_nodes

Array of peer nodes to contact when joining the network. Leave empty for a standalone or first node.

Type	Default
array of tables	[] (empty)

Each entry has the following fields:

Field	Type	Required	Description
address	string	✓	IP address of the bootstrap node
port	int	✓	Port of the bootstrap node

[[bootstrap_nodes]]
address = "192.168.1.100"
port = 5000

network_key

Hex-encoded AES-128 key used to encrypt peer-to-peer traffic. If null or omitted, peer communication is unencrypted.

Type	Default
string (32 hex characters)	null

network_key = "9f86d081884c7d659a2feaa0c55ad015"

Generate a key from the TUI (Network > Configure) or with the keygen command.

whitelist_ips (removed)

Dropped in favor of whitelist_node_ids. IP-based whitelisting has been removed. IPs are not a stable identity for a peer (they can be shared, spoofed, or reassigned), whereas node IDs are authenticated. Restrict peer communication with whitelist_node_ids instead. A whitelist_ips key left in an existing config file is ignored.

whitelist_node_ids

Peer node IDs this node will allow for communication. If configured, the node only accepts inbound DSN messages from these node IDs and only sends outbound messages to these node IDs.

Type	Default
array of strings	[] (allow all)

whitelist_node_ids = ["bootstrap-node", "trusted-worker-1"]

---

Environment Variables

These configure machine-local paths and runtime behavior. They are read directly from the process environment and are independent of the TOML configuration file described above.

LP_APP_DIR

Application configuration directory. Stores configs and credentials.

Default
~/.config/language_pipes

export LP_APP_DIR=~/.config/language_pipes

Directory structure:

app_dir/
├── configs/     # Configuration files
└── credentials/ # Credential files

LP_MODEL_DIR

Model cache directory. Stores downloaded model weights.

Default
~/.cache/language_pipes/models

export LP_MODEL_DIR=~/.cache/language_pipes/models

LP_NUM_LOCAL_LAYERS (deprecated)

Deprecated. Set num_local_layers per end model in the end_models configuration instead. This environment variable is still honored as a fallback default for end models that don’t specify num_local_layers, but it logs a deprecation warning and will be removed in a future release.

Number of initial model layers an end model executes locally before forwarding work to other nodes. Higher values improve prompt obfuscation by keeping more of the early pipeline on your machine. All nodes hosting the same end model should use the same value so that model layers are loaded correctly.

Default
1

export LP_NUM_LOCAL_LAYERS=1

LP_MAX_NODE_JOBS (deprecated)

Deprecated. Set max_node_jobs in the config file, or from the TUI’s “Jobs / Server” page, instead. This environment variable is still honored as a fallback default when max_node_jobs isn’t set in the config file, but it logs a deprecation warning and will be removed in a future release.

Maximum number of jobs this node will queue for a single peer node. Incoming jobs from a node whose queue is already full are rejected.

Default
10

export LP_MAX_NODE_JOBS=10

LP_MAX_API_JOBS (deprecated)

Deprecated. Set max_api_jobs in the config file, or from the TUI’s “Jobs / Server” page, instead. This environment variable is still honored as a fallback default when max_api_jobs isn’t set in the config file, but it logs a deprecation warning and will be removed in a future release.

Maximum number of pending jobs allowed per API key on the OpenAI-compatible API. Requests beyond this limit are rejected until earlier jobs for that key complete.

Default
5

export LP_MAX_API_JOBS=5

LP_8_BIT_MODE (deprecated)

Deprecated. Set data_type in the config file for each model, or from the TUI’s “Models / Layers” page when editing a model.
This variable is still honored as a fallback default for if a model does not have a data_type property in its configuration object.

Load model layers in 8-bit precision using the bitsandbytes library (LLM.int8 quantization) instead of the default 16-bit floating point. This roughly halves the memory needed for hosted layers at a small cost in output quality and speed.

The linear projection weights of each decoder layer are quantized to int8; the embedding, norms, and language-model head stay in float16.

Note: Requires the bitsandbytes package: pip install language-pipes[quantization] or pip install bitsandbytes.

Default
false

export LP_8_BIT_MODE=true

---

Hugging Face Authentication

hf_token

HuggingFace API token for downloading gated or private models (like Llama). This is stored separately from node configuration files, in <app_dir>/globals.toml. Language Pipes prompts for the token in the TUI when downloading a gated model and offers to save it for future downloads.

Get your token from https://huggingface.co/settings/tokens.

Note: For gated models (like Llama), you must also accept the model’s license agreement on the HuggingFace website before downloading.