By Ryze AI Team
This tutorial builds a question-answering app over your own documents using LangChain for orchestration and Chroma as the vector store. By the end you will have a working pipeline you can point at any text.
pip install langchain langchain-community chromadb sentence-transformers openai. Chroma runs embedded, so there is no separate server to start.
from langchain_community.document_loaders import TextLoader, PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
loader = TextLoader("data.txt")
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.split_documents(docs)
print(len(chunks), "chunks")
RecursiveCharacterTextSplitter tries to split on paragraphs, then sentences, then characters, which keeps chunks semantically coherent. Tune chunk_size and chunk_overlap to your content; 500 and 50 are reasonable defaults for prose.
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma
emb = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
db = Chroma.from_documents(chunks, emb, persist_directory="./chroma_db")
retriever = db.as_retriever(search_kwargs={"k": 4})
The embedding model runs locally, so no API key is needed for retrieval. Persisting to disk means you do not re-embed on every run.
from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
llm = ChatOpenAI(model="gpt-4o-mini")
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)
result = qa.invoke({"query": "What is the refund policy?"})
print(result["result"])
for d in result["source_documents"]:
print("source:", d.metadata)
Add an abstention instruction to the prompt so the model says it does not know when the context lacks the answer. Increase k or add a reranker if recall is poor. Log retrieved chunks alongside answers to diagnose misses. And evaluate on real questions before shipping.
If answers ignore the context, your prompt may not emphasize using it; add "Answer using only the provided context." If retrieval returns irrelevant chunks, check chunking quality and try a stronger embedding model. If Chroma is slow, ensure you are persisting and not rebuilding the index each run.