How to Use AnythingLLM

AnythingLLM packages private RAG, workspaces, and agents into a single desktop or Docker app.

AnythingLLM from Mintplex Labs is a batteries-included chat-with-your-documents platform. It handles ingestion, embedding, vector storage, LLM routing, and a clean workspace UI so you don’t have to wire five services together. You get a desktop build for personal use and a Docker image for team deployments.

What it is

A Node/React app that organizes knowledge into workspaces. Each workspace has its own documents, vector namespace, system prompt, and permitted users. It supports dozens of LLM providers (OpenAI, Anthropic, Ollama, LM Studio, Azure) and swappable vector stores (LanceDB by default, Pinecone, Qdrant, Chroma, Weaviate).

Install / set up

# docker
docker pull mintplexlabs/anythingllm
docker run -d -p 3001:3001 \
  --cap-add SYS_ADMIN \
  -v anythingllm_storage:/app/server/storage \
  mintplexlabs/anythingllm

First run

Hit http://localhost:3001, walk through onboarding, pick an LLM provider, and drop in an API key. Create a workspace, upload a few PDFs or point it at a website, and start chatting — embeddings happen in the background.

$ curl -F "file=@whitepaper.pdf" \
  http://localhost:3001/api/v1/document/upload
{"success": true, "location": "..."}

Everyday workflows

• Build a workspace per project or per client so contexts never leak between them.
• Enable the Agent tab to let the workspace call web search, scrape pages, or run SQL via custom skills.
• Expose the workspace as an embeddable chat widget on a marketing site with a scoped API key.

Gotchas and tips

The desktop app and the Docker server store data in different locations and don’t sync. Pick one deployment model up front. If you start on desktop and later move to Docker, you’ll re-ingest everything — there’s no migration path.

Default chunking is generic. For dense technical docs, bump chunk size and overlap in the workspace settings before ingesting, otherwise retrieval quality suffers. Re-embedding after changing chunking is a manual delete-and-reupload today.

Who it’s for

Small teams that need a private, multi-tenant RAG chat without building one. If you want ChatGPT-over-your-docs with user management and you don’t want to touch LangChain, AnythingLLM is the shortest path.