AI Medical Chatbot

General-purpose LLMs are fluent but not reliable for medical questions — they can produce confident, plausible-sounding answers that are wrong. I wanted to build a chatbot that answers from actual source documents, not from memory alone.

Designed and built the full RAG pipeline, from document ingestion to the served API.

The system uses retrieval-augmented generation (RAG): incoming questions are embedded and matched against a Pinecone vector index of source documents, the most relevant chunks are retrieved, and those chunks are passed to LLaMA (orchestrated via LangChain) as grounding context before it generates an answer. This means the model is answering from retrieved evidence rather than purely from its training data — a meaningful difference in a medical context where hallucination is costly.

I chose Pinecone for the vector store for its managed scaling and low-latency similarity search, which mattered for keeping response times reasonable in a Flask-served API. Redis caching was layered on top for repeated or similar queries.

Balancing retrieval precision (returning the right chunks) against response latency, and thinking through how to reduce hallucination risk when retrieved context was incomplete or ambiguous — an area I'd continue to invest in with better evaluation tooling.

Add an evaluation harness (retrieval precision/recall, answer groundedness scoring) rather than relying on spot-checking, and experiment with re-ranking retrieved chunks before generation.