How to use Text Generation Web UI (oobabooga) for local models
Text Generation Web UI is a browser-based interface for running local LLMs. It supports GGUF, GPTQ, and Transformers models, includes a chat UI and API server, and runs on any hardware. This guide covers installation and key features.
What Text Generation Web UI offers
Text Generation Web UI (commonly called 'oobabooga' after its creator) is a Gradio-based browser interface for local LLMs. It supports every major model format: GGUF via llama.cpp, GPTQ and EXL2 via ExLlamaV2, and full-precision models via HuggingFace Transformers.
Beyond basic chat, it includes: a character system for role-play with custom personas, a notebook mode for text completion without the chat wrapper, extensions for text-to-speech and speech-to-text, and a full API server compatible with both the OpenAI format and its own format.
Install on Windows, Mac, or Linux
Clone the repository: `git clone https://github.com/oobabooga/text-generation-webui`. Navigate into it and run the one-click installer for your platform: `start_windows.bat` (Windows), `start_linux.sh` (Linux), or `start_macos.sh` (macOS). The installer creates a conda environment and installs all dependencies automatically.
On first run, it will download PyTorch for your hardware (CUDA, ROCm, or CPU). The setup takes 5-15 minutes. After installation, the web UI opens at `http://127.0.0.1:7860`.
Load a model
Place GGUF model files in the `models/` directory. Go to the Model tab in the web UI, click the refresh icon, and select your model from the dropdown. For GGUF files, select 'llama.cpp' as the loader. Set the number of GPU layers to `99` to fully offload to GPU.
To download models from Hugging Face directly from the UI, use the Download tab. Enter the model repository name (e.g. `TheBloke/Mistral-7B-Instruct-v0.2-GGUF`) and select the specific file you want. Large downloads can be resumed if interrupted.
Use the API server
Enable the OpenAI-compatible API by starting with `--api` flag: `python server.py --api --listen`. This starts an OpenAI-compatible server at port 5000 alongside the web UI. Connect any OpenAI SDK client by setting `base_url='http://127.0.0.1:5000/v1'`.
The API supports streaming, system prompts, and temperature controls. Use it to connect local models to any application that uses the OpenAI SDK, LangChain, LlamaIndex, CrewAI, and others all work without code changes.
More Setup guides
How to install Ollama and run your first local model
Ollama is the fastest way to run open-source LLMs locally. This guide walks you through installation on macOS, Linux, and Windows, pulling your first model, and running it from the terminal or via its REST API.
Read guideHow to install LM Studio and load a model
LM Studio is a desktop app for running local AI models without any terminal commands. This guide covers installation, downloading a model from Hugging Face, and starting a local inference server.
Read guide