AnythingLLM
Chat with your documents using local or cloud models—desktop or self-hosted.
Quick facts
- Price model
- Freemium
- Starting price
- Free desktop; server pricing varies
- Best for
- Document Q&A · Private knowledge bases · Local RAG
- Replaces
- ChatPDF Pro, Notion AI, Enterprise RAG SaaS
- Platforms
- MacWindowsLinuxSelf-hosted
- Last verified
- 2026-06-22
Why it's listed
Ask questions of your files locally instead of uploading them to yet another AI subscription.
AnythingLLM ingests PDFs and notes into workspaces for RAG-style Q&A. Run the desktop app free or deploy server edition for teams without per-document SaaS metering.
The catch
Jump to setup guide ↓RAG quality depends on model choice and chunking; server features may be paid.
How to set up AnythingLLM
Chat with your PDFs and notes locally—desktop app first, no per-document ChatPDF subscription.
- Time
- 25–40 min
- Difficulty
- Moderate
- Verified
- 2026-06-22
Before you start
- Desktop app from anythingllm.com (Mac/Windows/Linux)
- Folder of PDFs or docs to test
- Ollama installed for free local models, or an API key for harder questions
Install desktop edition
Download AnythingLLM Desktop—not the server edition unless you already run Docker. Launch and skip cloud signup if offered.
Create a workspace
New workspace → name it (Taxes 2024, Lease docs, Research). Workspaces isolate embeddings and chat history.
Connect a model
Settings → LLM Provider: Ollama (localhost) for free local chat, or OpenAI/Anthropic with your BYOK key. Test a hello prompt.
Upload documents
Drag PDFs, txt, or markdown into the workspace. Wait for embedding to finish—status shows in UI. Start with 3–5 files, not 500.
Ask grounded questions
Chat with citations enabled. Ask specifics: dates, dollar amounts, clause text. If answers hallucinate, add more source files or switch to a stronger model.
Optional — Docker server for household
Server edition on homelab for shared workspaces. Heavier setup—desktop is enough for solo document Q&A.
Troubleshooting
- Embeddings fail
- File too large or scanned PDF without OCR—run OCR first or split PDF.
- Answers ignore documents
- Wrong workspace selected, or embedding not complete. Check document list shows green processed state.
- Slow on laptop
- Smaller Ollama model for chat; reduce chunk count in advanced settings.
Keep it working
- Remove old workspaces you do not need—saves disk
- Update desktop app when prompted
- Sensitive docs stay local—still back up the AnythingLLM data folder
Official docs: docs.anythingllm.com/installation-desktop/overview
Good fit for
- Researchers
- Small firms with document archives
Not ideal for
- Huge enterprises needing managed compliance out of the box
Alternatives
Open WebUI
Self-hosted web interface for chatting with local or BYOK-connected models.
Replaces: ChatGPT Team, Custom GPT enterprise tiers
Local AI Starter Guide
Step-by-step path from zero to running useful local models on a typical home computer.
Replaces: AI SaaS for basic drafting, Paid local AI courses
LM Studio
Run local LLMs on your Mac or PC with a friendly desktop app—no API subscription required.
Replaces: ChatGPT Plus, Claude Pro…
Ollama
CLI and API for running open models locally with simple pull-and-run commands.
Replaces: OpenAI API for local tasks, Hosted LLM sandboxes