AnythingLLM combines model connections, document retrieval and workspaces. It can help search a small set of manuals or internal notes, with separate workspaces for different projects. Retrieval-augmented generation (RAG) supplies relevant document fragments to a model; it does not guarantee a correct answer or automatically train the model.
Target: Ubuntu Server 24.04 LTS, x86-64, SSH and a sudo account. Documentation reviewed 15 November 2026. Commands have not been tested end to end on Ubuntu.
Before you start
Allow roughly 2–4 vCPUs, 4–8 GB RAM and 20 GB free disk for a small document workspace using an external model API. Local generation and local embeddings need additional RAM/CPU. Choose both a chat provider and an embedding provider; their privacy and billing settings are separate.
Resource figures are planning estimates for a small evaluation, not performance guarantees or plan specifications. A VPS/VDS does not imply a GPU. Check available memory, storage and resources before ordering.
On a fresh Ubuntu installation, use the packages below. If Docker is already installed from another repository, keep that installation and check compatibility first: do not mix Docker CE and docker.io packages. Use Compose v2; if the package is missing, check that Ubuntu Universe is enabled.
sudo apt update
sudo apt install -y docker.io docker-compose-v2 curl openssl
sudo systemctl enable --now docker
sudo docker compose version
All examples publish ports only on the server’s 127.0.0.1. Access them from your computer through an SSH tunnel; a domain is unnecessary. Do not open these ports to the Internet. An internal container listener on 0.0.0.0 does not change the host’s loopback-only published port.
Each project below is a new installation. Do not rerun secret-generation steps against an existing deployment.
1. Persist storage and application settings
Create writable storage for the container’s UID 1000 and a persistent settings file. Save the configuration as compose.yaml. This basic text/document recipe does not enable privileged browser capabilities.
umask 077
mkdir -p ~/ai-anythingllm
cd ~/ai-anythingllm
sudo install -d -m 700 -o 1000 -g 1000 storage
sudo touch storage/.env
sudo chown 1000:1000 storage/.env
sudo chmod 600 storage/.env
services:
anythingllm:
image: mintplexlabs/anythingllm:latest
restart: unless-stopped
ports:
- "127.0.0.1:3001:3001"
environment:
STORAGE_DIR: /app/server/storage
volumes:
- ./storage:/app/server/storage
- ./storage/.env:/app/server/.env
2. Open the setup privately
Start on the VPS and open a tunnel from your own computer. Browse to http://127.0.0.1:3001. Complete setup, choose password protection or multi-user access and keep the workspace private. Select providers in the application instead of putting API keys in public scripts.
sudo docker compose up -d
sudo docker compose ps
curl --fail-with-body http://127.0.0.1:3001/
ssh -N -L 127.0.0.1:3001:127.0.0.1:3001 YOUR_USER@YOUR_SERVER_IP
3. Verify retrieval with a controlled document
Create a plain-text sample containing a unique fictional project name and a short fact. Upload it, add it to one workspace and wait for embedding/indexing. Ask for the fact and inspect the retrieved passage. Ask an unrelated question too: missing evidence should not become an invented citation. Review the source passage before using an answer at work.
4. Understand where data goes
An external chat model can receive retrieved text, and an external embedding provider can receive document chunks. Self-hosting the UI does not keep those requests local. Start without agents, connectors or browser scraping; add only capabilities needed for the use case. Back up the entire storage directory, including its hidden .env and secrets, while the application is stopped. Removing a document from one workspace is not a complete retention policy for uploads, logs, exports and backups.
Updates and recovery
Before upgrading, stop the application for a consistent backup, keep data and secrets off the VPS, and test recovery on a separate instance. Moving image tags in these examples are for initial evaluation; pin a reviewed release tag or digest before ongoing use. Record the current digest with sudo docker image inspect IMAGE_NAME --format '{{json .RepoDigests}}'. Avoid unattended upgrades without compatibility checks, and do not delete persistent volumes with down -v.
If startup fails, inspect sudo docker compose ps, recent logs, free disk and memory. Do not share logs containing keys, prompts or personal data. For directory-permission errors, check the expected UID/GID and ownership instead of granting 777. Set up HTTPS, authentication and request limits before sharing access.