How AI Lets You Chat With PDF Documents: A Practical Guide to RAG

The problem: Ctrl+F only finds words, not meaning

A search box finds the literal string you typed. It won't find a clause about "termination rights" if the contract calls it "early exit," and it can't answer "what was the main driver of the loss in Q3?" — because that answer is a synthesis of several sentences, not one, findable string. Once a document is more than a few pages, skimming stops working and you need something that understands what you're asking, not just what you typed.

What Retrieval-Augmented Generation actually does

Retrieval-Augmented Generation (RAG) is a simple idea with a complicated name: instead of asking a language model to answer from memory (where it can guess, misremember, or "hallucinate"), you first retrieve the exact passages of your document that are relevant to the question, and then ask the model to answer using only those passages. The model isn't reciting facts it memorized during training — it's reading a stack of excerpts you handed it a moment ago and summarizing what they say.

That distinction is what makes the citations possible. Because the answer is built from specific excerpts, the system already knows which page each excerpt came from — so it can show you exactly where to verify the claim.

What happens when you upload a PDF

Four steps run in the background the moment you upload, and you can keep working while they finish:

1. Text extraction, page by page. The PDF is parsed with PyMuPDF, and the text of every page is pulled out separately so the page number is never lost. Pages with no extractable text (common in purely scanned documents) are skipped, and if a document has no text anywhere, it's marked as needing OCR rather than silently producing empty answers.
2. Chunking. Each page's text is split into overlapping chunks — small enough for accurate retrieval, large enough to keep sentences intact, and never crossing a page boundary so every chunk keeps a single, correct page number attached.
3. Embedding. Every chunk is converted into a vector — a list of numbers that captures its meaning — using an embedding model, in batches rather than one call per chunk.
4. Indexing. The vectors are stored in a vector database (Qdrant) alongside the chunk's text, page number, and the ID of the document and user it belongs to, so later searches can be filtered to exactly one document and one account.

What happens when you ask a question

Your question goes through a similar, much faster pipeline:

1. Your question is converted into a vector using the same embedding model.
2. That vector is compared against every chunk vector belonging to that specific document and your account — never anyone else's — and the closest matches are retrieved.
3. For documents that fit comfortably in the model's context window (most reports and papers), the entire document is given to the model directly, which is what makes aggregate questions like "how many references does this paper cite?" answerable — no handful of excerpts could ever contain that answer, only the full text can. Larger documents fall back to just the retrieved excerpts.
4. The model answers strictly from that content, under a system prompt that tells it not to invent facts, to say so if the document doesn't contain the answer, and to ignore any instructions embedded in the document text itself (a real safety concern, since the "context" is just untrusted text from a file you uploaded).
5. The answer comes back with the page number and excerpt behind it, so you can check it in seconds.

Why the citations are the actual point

Language models are fluent, which makes an ungrounded wrong answer just as confident-sounding as a correct one. The fix isn't asking the model to "be more careful" — it's changing what you're able to check. A page number and an excerpt turn "trust me" into "here, read it yourself," which is the only way an AI answer about a document you care about should be treated.

Where this is genuinely useful

• Research papers — ask about methodology, results, or find every mention of a specific term without hunting through the PDF manually.
• Contracts — locate obligations, deadlines, and termination clauses by what they mean, not just what they're literally called.
• Financial and board reports — ask about a specific quarter's numbers without scrolling past pages of formatting.
• Manuals and SOPs — get the one relevant procedure instead of reading the whole document to find it.
• Textbooks and long reference PDFs — ask a question the way you'd ask a tutor, and get pointed at the right page.

Getting better answers

• Ask one focused question at a time rather than several bundled together.
• Use wording closer to the document's own terms if a broad question comes back empty — it helps retrieval find the right passages.
• Always check the cited page when the answer matters — that's what the citations are for.

Frequently asked questions

Does PDF Chat work with scanned PDFs?

Only if the PDF already contains a text layer. Purely scanned, image-only pages have no extractable text, so the document is marked failed with a message that OCR is required. OCR support is planned as an extension to the existing extraction pipeline.

Can other users see my documents or ask questions about them?

No. Every document, chunk, and chat message is tied to your account. Every vector search and every database query is filtered by the authenticated user's ID, so no one can retrieve or query another user's files.

Why does it sometimes say the document doesn't contain the answer?

The system prompt instructs the model to answer only from the supplied document content and say so plainly if the answer isn't there, rather than guessing. For very large documents that fall back to excerpt-based retrieval, try rephrasing your question using wording closer to what the document actually says.

How large a PDF can I upload?

The default limit is 20 MB per file, which is configurable by the person running the deployment. Very large documents automatically switch from full-document mode to top-k excerpt retrieval so answers stay fast.