RAG-Anything is an open source Python framework for chatting with your PDFs and Office documents without losing their images, tables, or equations. Instead of flattening the file to plain text, it separates each element, describes it with a language or vision model, and connects everything into a knowledge graph you can query in natural language. It’s developed by HKUDS, the lab at the University of Hong Kong behind LightRAG, has an MIT license, and has over 24,100 stars on GitHub.
What is RAG-Anything?
RAG-Anything is a multimodal RAG pipeline built on top of LightRAG. It covers the entire journey, from document ingestion to response, in four stages:
- Parsing. A parser divides the file into blocks of text, images, tables, and equations, and records which page each one comes from.
- Type-based analysis. Each element goes through its own processor. A vision model writes descriptions of the images taking context into account, tables are interpreted as structured data, and equations are processed in LaTeX.
- Knowledge graph. Elements are converted into entities related to the text around them. A chain of
belongs_torelationships preserves the hierarchy of sections in the document. - Hybrid retrieval. Queries combine vector search with graph traversal, so an answer can bring a paragraph together with the chart that explains it. It’s the idea behind
knowledge graph RAGorgraph RAG, extended to non-text content.
It accepts PDFs, Office documents (DOC, DOCX, PPT, PPTX, XLS, XLSX), images (JPG, PNG, BMP, TIFF, GIF, WebP), and text files (TXT, MD). Recent versions also accept audio and video, which we cover below.
Why does traditional RAG fail with PDFs that have tables and charts?
Because a typical RAG pipeline extracts the text layer from the PDF, chunks it, and generates embeddings of those chunks. Everything that isn’t running text gets lost along the way:
- A bar chart disappears.
- A table becomes a list of numbers without its column headers.
- A formula ends up as broken characters.
In technical documentation, financial reports, and research papers, the answer is usually right there.
RAG-Anything treats those elements as first-class content. A chart is described by a vision model and saved as an entity linked to the paragraph that mentions it. A table keeps its structure and its caption.
The approach is described in the authors’ technical report on arXiv (2510.12323, October 2025). According to the authors themselves, it outperforms existing methods on multimodal benchmarks, with the biggest gains on long documents. These are results reported by the team that created the tool, and the report is a preprint.
What’s the difference between LightRAG and RAG-Anything?
LightRAG is the engine and RAG-Anything is the multimodal layer on top. LightRAG provides the graph-based indexing, storage, and query modes. RAG-Anything adds the parsers, type-specific processors, and relationships between text, images, tables, and equations within the graph.
The two projects are very connected:
- If you already have a knowledge base in LightRAG, you can pass that instance to RAG-Anything. It inherits the working directory, the LLM function, and the embeddings function, and you can add multimodal documents to the same storage.
- Since June 2026, LightRAG itself enables multimodal RAG through a native integration of RAG-Anything.
Rule of thumb: if your documents are mostly prose, LightRAG is enough. If the answers are in figures, tables, or formulas, you need the RAG-Anything layer.
RAG-Anything or MarkItDown?
It depends on whether you need a complete RAG system or just convert documents to pass to a model. MarkItDown is a lightweight Microsoft utility that converts PDFs, Office, images, audio, HTML, and EPUB to Markdown, preserving headings, lists, tables, and links. It’s designed to feed LLMs, but only converts: it doesn’t index or retrieve anything.
| MarkItDown | RAG-Anything | |
|---|---|---|
| What it produces | A Markdown file | A queryable knowledge graph |
| Images | EXIF and OCR; optional LLM descriptions for PPTX and images | Vision model description of each image, linked to text |
| Search and answers | Not included | Hybrid vector and graph retrieval |
| Models needed | None for basic conversion | LLM, vision model, and embeddings |
| Installation | pip install 'markitdown[all]' |
pip install raganything |
If you’re going to paste a document into Claude Code’s context or your agent, MarkItDown is the most direct tool. If you have a folder with hundreds of documents and want to ask them questions whose answers are in charts and tables, RAG-Anything is the way to go.
How do you install RAG-Anything?
It installs from PyPI and requires Python 3.9 or higher:
pip install raganything
# Optional extras for more formats
pip install 'raganything[all]'
pip install 'raganything[image]' # BMP, TIFF, GIF, WebP
pip install 'raganything[text]' # TXT, MD
If you prefer to work from source, the project uses uv:
git clone https://github.com/HKUDS/RAG-Anything.git
cd RAG-Anything
uv sync
uv sync --all-extras # all optional features
There are two things pip won’t install for you:
- LibreOffice is required to process Office documents. On macOS it’s
brew install --cask libreoffice, on Ubuntu/Debiansudo apt-get install libreoffice, and on Windows, the installer from the official site. - MinerU models download automatically the first time you process a document.
To check that the parser is ready:
mineru --version
MinerU, Docling, or PaddleOCR: which parser should you use?
Start with MinerU, which is the default parser, unless your corpus is mostly Office: in that case try Docling. RAG-Anything supports three parsers and you choose one with the parser argument or the PARSER environment variable:
| Parser | Best for |
|---|---|
| MinerU (default) | PDFs, images, and Office; solid OCR and table extraction; GPU acceleration |
| Docling | Office documents and HTML; better document structure preservation |
| PaddleOCR | Heavy OCR work on scanned PDFs and images; converts Office/TXT/MD to PDF first |
PaddleOCR needs its own extra (pip install 'raganything[paddleocr]') and also a paddlepaddle version for your platform, installed by following its official guide.
The parsing method can be auto, ocr, or txt. MinerU also accepts options like device (cpu, cuda, mps), a page range, and lang for OCR.
How do you chat with your PDFs using RAG-Anything?
You configure three model functions (an LLM, a vision model, and an embeddings function), process the file, and query it. The main calls are these:
config = RAGAnythingConfig(
working_dir="./rag_storage",
parser="mineru",
parse_method="auto",
enable_image_processing=True,
enable_table_processing=True,
enable_equation_processing=True,
)
rag = RAGAnything(
config=config,
llm_model_func=llm_model_func,
vision_model_func=vision_model_func,
embedding_func=embedding_func,
)
await rag.process_document_complete(
file_path="path/to/your/document.pdf",
output_dir="./output",
parse_method="auto",
)
result = await rag.aquery(
"What are the main findings shown in the figures and tables?",
mode="hybrid",
)
The full definitions of the three model functions are in the README. The official example connects them to the OpenAI API, and the base_url parameter lets you point them to any compatible endpoint.
There are three ways to query:
- Text queries, with LightRAG modes:
hybrid,local,global, andnaive. - VLM-enhanced queries, which activate automatically when you have a vision model configured. If the retrieved context includes images, the images themselves are sent to the vision model, not just their descriptions.
- Multimodal queries (
aquery_with_multimodal), which let you include a table or an equation in LaTeX in the question itself.
To process an entire folder, process_folder_complete takes a list of extensions, a recursive flag, and max_workers.
If you’ve already processed your documents with another tool, insert_content_list accepts already-parsed content and skips parsing entirely.
The repository includes a ready-to-use end-to-end script:
python examples/raganything_example.py path/to/document.pdf --api-key YOUR_API_KEY --parser mineru
Can RAG-Anything process audio and video?
Yes, since versions 1.4.x (September 2026). Both functions are optional.
- Audio (.mp3, .wav, .flac, .m4a, .ogg)
- Install with
pip install 'raganything[audio]'and enableenable_audio_processing=True. - Transcription is done locally with faster-whisper, using the model you specify in
WHISPER_MODEL(default isbase).
- Install with
- Video (.mp4, .mov, .webm, .avi, .mkv)
- Install with
pip install 'raganything[video]', which also needsffmpegin your PATH, and enableenable_video_processing=True. - The pipeline detects scenes with SceneDetect, describes key frames with the vision model, transcribes the audio track, and stitches it all together by timestamp.
- Install with
Media files enter the same knowledge base as your documents. So a single question can draw on both the PDF specification and a recording of a design review.
Is RAG-Anything free?
The framework is free and MIT-licensed, but running it isn’t necessarily free, because it calls three models: an LLM, a vision model for the images, and an embeddings model. If you use hosted APIs, you pay per token at indexing time, since each image and table goes through a model, and again at query time.
Some parts run locally at no API cost:
- Parser models (MinerU, Docling, PaddleOCR).
- Audio transcription with faster-whisper.
- The parsing test scripts included (
office_document_test.py,image_format_test.py,text_format_test.py), which exercise the parser without an API key.
That’s why they’re a good way to check how well your documents parse before spending money on indexing.
What about you? What documents would you like to be able to query with a multimodal RAG in your work?
- MinerU
- Docling
- PaddleOCR
- I’d prefer another option (tell us which)
Comment below or, if this article reached you by email, reply directly to the email: your reply gets published here.
Related: Building the Entire RAG Ecosystem and Optimizing Every Component