kyleg142/localsage

🔮 Local Sage

A human-in-the-loop LLM interface that embraces the command line.

Local Sage running in Zellij, alongside Helix and Yazi.

About 🔎

Local Sage is an open-source stateful CLI for interacting with LLMs in shell-based workflows.

Featuring live Markdown rendering with inline math conversion for a silky smooth chatting experience. Designed to hook into any OpenAI API endpoint, and tested with local LLMs hosted via llama.cpp.

Additional Features:

• Standard Output Rendering: History is persistent and scrollable, even after exiting the CLI.

• Standard Input Piping: Pipe stdin directly into Local Sage from the command line or a script.
  Example: ps aux | localsage "What process is consuming the most memory?"

• Fancy Pro*s**: Command completion, path completion, and in-memory history for a shell-native UX.

• Website Scraping: Scrape a website with a simple command, and attach it's contents to the current session.

• Context-aware Attachment: Attachments are replaced on re-attachment and can be purged from a session, restoring context.
  Note: Entire directories can be attached at once. Attached website content can be purged from a session as well.

• Automated Environment Context: Your model sees your username, your OS name, and the contents of your current working directory.
  Note: When changing the working directory with the !cd command, your model is aware. Environment context mutates.

• Session Management: Load, save, delete, reset, and summarize sessions.

• Profile Management: Save, delete, and switch model profiles.

• Context & Throughput Monitoring: Shown through a subtle status panel.

• Built-in Markdown themes: Customize your output with a variety of built-in Markdown themes. Available themes are listed here.

Check out the Under the Hood section if you want to learn more!

Compatibility 🔩

Python 3.10 or later required.

The big three (Linux, macOS, and Windows) are all supported. Ensure your terminal emulator has relatively modern features. Alacritty works well. So does kitty and Ghostty.

You can use non-local models with Local Sage if desired. If you set an API key, the CLI will store it safely in your OS's built-in credential manager via keyring. Setting the OPENAI_API_KEY environment variable works as well, and overrides all other options.

Each release is automatically published to Docker Hub. You can view instructions for running the CLI in Docker here.

Installation 💽

Install a Python package manager for your OS. Both https://github.com/astral-sh/uv and https://github.com/pypa/pipx are highly recommended.

For uv, open your terminal and type:

bash
uv tool install localsage

Or, for pipx, type:

bash
pipx install localsage

Type localsage into your terminal to launch the CLI. Type !h to view command usage.

Getting Started ✔️

Configuration is done entirely through interactive commands. You never have to touch a config file.

1. Configure a profile with !profile add. API endpoint format: http://ipaddress:port/v1.
2. Type !profile switch to switch to your new profile.
3. Use !ctx to set your context length.
4. (Optional) Set your own system pro*** with !prompt or an API key with !key.

[!TIP] If you press tab while at the main pro***, you can access a command completer for easy command use.

Dependencies 🧰

Local Sage is designed with minimal dependencies, keeping the download light and minimizing library bloat.

• https://github.com/Textualize/rich - Used extensively throughout. Panels, live rendering, etc.
• https://github.com/pro***-toolkit/python-pro***-toolkit - Pro***s and completers, also used extensively.
• https://github.com/openai/openai-python - Provides all API interaction as well as the conversation history list.
• https://github.com/jaraco/keyring - Safely handles API keys on all platforms.
• https://github.com/openai/***en - Provides tokenization and enables context length calculation.
• https://github.com/platformdirs/platformdirs - Detects default directories across operating systems.
• https://github.com/phfaist/pylatexenc - Absolutely vital for live math sanitization.
• https://pypi.org/project/pyperclip/ - For copying code blocks to the system clipboard.
• https://pypi.org/project/trafilatura/ - For extracting text from web pages.

File Locations 📁

Your config file, session files, and error logs are stored in your user's data directory.

Commands 📄

All usage charts are present below. You can render them within the CLI by typing !h.

---

---

---

Docker 🐋

Pulling from Docker Hub and running the container:

bash
# Download the image
docker pull kyleg142/localsage

# Run the container, '-e OPENAI_API_KEY' is optional
docker run -it --rm \
  -v /pick/a/data/directory:/root/.local/share/LocalSage \
  -e OPENAI_API_KEY \
  kyleg142/localsage

Piping is not supported when running Local Sage in a container.

Notes on Networking

You may have to add specific options to your docker run command if you are running a non-containerized backend/API on the same machine.

Local Linux

1. Add --network host to your docker run options to allow the container to reach services on localhost.
2. Follow the Getting Started section above.

Local Windows/Mac

1. Add --add-host=host.docker.internal:host-gateway to your docker run options.
2. Run the container, type !profile add to create a new profile. Set the API endpoint to http://host.docker.internal:8080/v1 when pro***ed.
3. Ensure your API endpoint (llama.cpp, vllm, etc.) is listening on 0.0.0.0:8080.

Display Notes 🖥️

Typing into the terminal while streaming is active may cause visual artifacting. Avoid typing into the terminal until the current generation finishes.

A monospaced Nerd font is HIGHLY recommended. It ensures that Markdown, math, and icons all align well on-screen. The root pro*** uses a Nerd font chevron.

Under the Hood 🛠️

Context-Aware Attachment

When a file (or directory) is attached to a session, a wrapper is applied to the contents before being appended to the session history. This wrapper enables attachment detection via regex, for file management.

If you re-attach a file, context consu***ion is massively reduced by removing the entry containing the file contents from the session history and then appending the new copy. You can also completely remove a file from a session via the !purge command, which restores context spent.

A similar wrapper is applied to website content. That means !purge can also be used to remove scraped website content from the conversation history.

Automated Environment Awareness

Your model is provided with basic environment context that mutates depending on the current working directory.

This is what your model can see...

shell
[ENVIRONMENT CONTEXT]                        # Header
Current User: {USER_NAME}                    # Your username
Operating System: {system_info}              # Your operating system
Working Directory: {wd}                      # Current working directory, mutates when using !cd
Visible Files: {", ".join(files[:20])}       # Up to 20 visible file names, mutates when using !cd
Visible Directories: {", ".join(dirs[:20])}  # Up to 20 visible directory names, mutates when using !cd

Rendering & Streaming

At its core, Local Sage uses the Rich library com***ed with a custom math sanitizer to render live Markdown and readable inline math. Chunk processing is frame-synchronized to the refresh rate of a rich.live display, meaning that the entire rendering process occurs at a customizable interval. Effectively a hand-rolled, lightweight, synchronized rendering engine running right in your terminal.

You can adjust the refresh rate using the !rate command (30 FPS by default).

Limitations 🛑

Once the live panel group fills the terminal viewport, real-time rendering cannot continue due to terminal constraints. By default, the Response panel consumes the Reasoning panel to conserve space (toggleable with the !consume command).

This should only be noticeable on large responses that consume over an entire viewport's worth of vertical space.

Local Sage is text-only. This limitation keeps Local Sage portable, lightweight, and backend-agnostic.

Versioning 🔧

The project follows basic versioning:

• 1.0.x - Minor patches consisting of bug fixes and tweaks
• 1.x.0 - Major patches consisting of large feature expansions or necessary refactors.

License ⚖️

Local Sage is released under the https://opensource.org/license/mit.

AI Use 🤖

I use AI for theory-crafting, code review, and snippet generation. For example, most of the regex seen in math_sanitizer.py was generated and tuned by iteratively pro***ing GPT 5.1 and then ran against a pytest suite to prevent regression. The architecture of Local Sage is written and tuned by hand, and all documentation you see here is written by hand as well. This is NOT a 'vibe-coded' project.

Closing Notes 🫵

Local Sage is an open-source, single-dev project built purely for the love of the game. Please be kind!

Contributions are always welcome! See https://github.com/Kyleg142/localsage/blob/main/.github/CONTRIBUTING.md.