Build a document Q&A bot

By Ryze AI Team

This tutorial builds a bot that answers questions over a set of PDFs, combining extraction, chunking, embedding, retrieval, and generation.

1. Extract text

from pypdf import PdfReader
reader = PdfReader("doc.pdf")
text = "\n".join(p.extract_text() for p in reader.pages)

For scanned PDFs, use OCR first with pytesseract or a cloud OCR service.

2. Chunk with metadata

from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=40)
chunks = splitter.split_text(text)
# store page numbers as metadata for citations

3. Embed and store

from sentence_transformers import SentenceTransformer
import numpy as np
model = SentenceTransformer("all-MiniLM-L6-v2")
embs = model.encode(chunks)
# store embs, chunks, and metadata in a list or vector DB

4. Query pipeline

q_emb = model.encode([question])
scores = np.dot(embs, q_emb.T).flatten()
top = np.argsort(scores)[-4:][::-1]
context = "\n\n".join(chunks[i] for i in top)
prompt = f"Answer using only this context:\n{context}\n\nQuestion: {question}"

Send the prompt to your LLM with an abstention instruction.

5. Cite sources

Include page numbers in the answer and let users click to open the source document. Citations are the main trust mechanism for document Q&A.

6. Tips

Chunk size matters more than model choice for PDFs. Preserve table structure where possible. Re-index when documents change. And evaluate on real questions from your users.