Skip to content
$ Language Pipes

Release Notes

Release 2.0.0

TUI

Major version change of Language Pipes, we got a TUI!

        _                                                   ____   _
        | |                                                 |  __`\(_)
        | |     __ _  ___   ___  _   _  __ _  __ _  ___     | |__) | |_ __   ___  ___
        | |    / _` |/ _ \ / _ `| | | |/ _` |/ _` |/ _ \    |  ___/| | '_ \ / _ \/ __|
        | |___| (_| | | | | (_| | |_| | (_| | (_| |  __/    | |    | | |_) |  __/\__ \
        |______\__,_|_| |_|\__, |\__,_|\__,_|\__, |\___|    |_|    |_| .__/ \___||___/
                            __/ |             __/ |                  | |
        Version 2.0.0      |___/             |___/                   |_|


                                   |> New Configuration <|


                                      Load Configuration


                                            Exit




        Arrows U/D: Move                   Enter: Select                   Esc: Back

The tui adds the ability to start and stop models on demand, change the configuration on the fly, and more. It’s broken down into a few sections:

  • Home / Dashboard: main status screen that allows you to start and stop the network servers as well as show the status of many different parts of the language pipes program.
  • Home / Activity: Displays any logs for the network server, job server, or status of the models
  • Network Pages: Change how the peer to peer network is configured for your device. Has interactive setup and you can start or stop the server at any time.
  • Model Pages: Allows you to configure layer or end models to load on demand.
  • Pipe Pages: Shows the status of the current pipes you are connected to or are on your network.
  • Job Pages: Allows configuration of the job server and show the status of active jobs.

See the updated CLI configuration guide for more information on advanced usage including a run mode that just prints to std out.

Want to use the same tech to make your own TUI? Check out my ansinout library!

Responses API

Added support for /v1/responses of the OpenAI compatable server. Works with any harness or web UI that supports the responses endpoint.

from openai import OpenAI


client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="foo"
)


response = client.responses.create(
    model="Qwen/Qwen3-1.7B",
    instructions="You are a helpful assistant.",
    input="What is distributed computing?",
    max_output_tokens=200
)


print(response.output_text)

Gemma 4 Support

Added support for Google’s newest language model line. Tested Models:

  • google/gemma-4-E2B-it
  • google/geema-4-E4B-it
  • More tests in the future, but the program should support any Gemma 4 model!

Release 1.2.0

—api-keys flag

Added flag to denote API keys to use for OpenAI compatible server. This flag can have many values just like --layer-models so use like --api-keys foo bar baz and then for the request add a ‘Authorization’ header with a value like Bearer foo.
See official documentation for more information

Small changes

  • Added a version checker to determine the latest version. It sends a request to raw.github.com for the pyproject.toml file and reads to version property.
  • Added gemma to model support document
  • Self upgrading is not a good pattern so I removed the language-pipes upgrade command. Use pip install language-pipes --upgrade to upgrade the package
  • Don’t default to yes for OpenAI server config

Release 1.1.0

Model Support

Added model support for GLM4.1v, Gemma3, and Phi4 model families. Below is a list of newly tested models, the same model but . You can view current model support at the model support page. Models of the same family but in different sizes should also work.

Phi:

  • microsoft/Phi-4-mini-reasoning
  • microsoft/Phi-4-reasoning-plus

Z.ai:

  • zai-org/GLM-4.1V-9B-Thinking

Meta:

  • meta-llama/Llama-3.1-8B-Instruct
  • meta-llama/Llama-3.2-1B-Instruct

Gemma:

  • google/gemma-3-1b-it

Whitelists

—whitelist-ips if set, only network IPs that are in this list are allowed to communicate with this node —whitelist-node-ids if set, only node IDs that are in the list are allowed to communicate with this node

Note: --whitelist-node-ids works by ECDSA signature. Public keys are stored in ~/.config/language_pipes/credentials and returning nodes must match the same public signature to be allowed to use their node ID.

Bug fixes

  • Asks for huggingface API token via text prompt if not in environment variable.
  • Models layers don’t have to line up exactly to complete a pipe, this should make num_local_layers easier to work with.
  • v1/models endpoint now only includes a model if all of the layers are available in a pipe AND the node hosting the open AI compatible server has an end model for that model ID
  • Added more stop tokens from the config. Llama3 does not use the same place so we pass a set of tokens as stop tokens instead of just config.eos_token_id.

Release 1.0.0

Features:

Huggingface hub
We are now using the huggingface_hub python library to download models. This allows you to download gated models by supplying it an api key either through the LP_HUGGINGFACE_TOKEN environment variable or through input whenever a model needs to be downloaded.

—end-models
Added a --end-models flag for setting end models to run, also available as a configuration setting or environment variable. This changes the configuration from the --hosted-models flag to separate --layer-models and --end-models flags. You can use the flag as follows

Terminal window
language-pipes serve --config config.toml --end-models [model name]

—num-local-layers Adds the --num-local-layers flag for a new layer of security for end models. Also available in a configuration setting or environment variable. This setting lets you set how many layers an end model will run. This will mitigate SipIt attacks (see the write up here). Important this setting must be the same for all nodes on the network. You can use the flag as follows:

Terminal window
language-pipes serve --config config.toml --num-local-layers 5

AES keys We now support rolling AES IVs to prevent predictable IV attacks

Layer size computation
Uses a better algorithm for detecting the size of the weights inside of a layer.

Usability
The project is using the keyboard interrupt (ctrl+c) as a “go back” button to go back to a previous menu. This has been overhauled and more places should allow you to go back to the last menu instead of exiting the program or going back to the main menu. You can always use the ctrl+d interrupt to kill the process.

Logging
Added the date and time to the logger for better logs on long running servers.

Notes

I’m finally happy with the way the project is laid out and what it is trying to accomplish. Privacy for layer nodes is a much more complicated problem that I first realized so I have updated the privacy wording in the documentation to match a probabilistic threat model instead of assuming hidden states can never be reversed. I will be doing case studies in hidden state reversal and finding mitigation techniques that give the user more trust that their prompts will not be compromised.