IJCATR Volume 15 Issue 1

LocalRAG: A Privacy-Preserving Offline Framework for Multi-PDF Question Answering

Dr. Ranga Rao Velamala
10.7753/IJCATR1501.1003
keywords : Local RAG, Retrieval-Augmented Generation, Large Language Models, Document-Grounded QA, Hybrid Retrieval.

PDF
The exponential growth of PDF documents across public and private organizations has made manual document review increasingly labor-intensive, driving demand for automated, document-grounded question-answering systems. Such systems require robust text extraction from PDFs. This often involves optical character recognition (OCR) for scanned documents, followed by retrieval-augmented generation (RAG). However, most existing RAG implementations depend on commercial Large Language Model (LLM) APIs. This dependence introduces high operational costs and latency, limiting suitability for real-time use, and raises significant privacy and security concerns, particularly in regulated sectors including finance, healthcare, and education. To address these challenges, this paper introduces LocalRAG, a fully local and offline RAG framework implemented as a standalone Python and Streamlit application that operates entirely on CPU and does not require external services. The framework processes PDFs using PyMuPDF, with optional integration of Tesseract OCR for scanned documents. LocalRAG employs a hybrid retrieval approach that combines dense vector embeddings indexed with FAISS (BGE-small-en-v1.5) and sparse keyword-based retrieval using BM25. The system utilizes a quantized four-billion-parameter instruction-tuned language model (Qwen3 4B Instruct), enabling efficient inference on modest consumer hardware. To ensure privacy preservation, source metadata is systematically removed from the LLM context, which mitigates the risk of sensitive information leakage and supports administrative traceability via structured JSON and CSV exports. Experimental evaluations conducted on standard consumer hardware indicate low-latency responses, high answer accuracy, and outputs that are well-grounded in source documents. LocalRAG thus provides a privacy-preserving, reproducible, and practical baseline for deploying local RAG systems in organizational and regulated environments
@artical{d1512026ijcatr15011003,
Title = "LocalRAG: A Privacy-Preserving Offline Framework for Multi-PDF Question Answering",
Journal ="International Journal of Computer Applications Technology and Research (IJCATR)",
Volume = "15",
Issue ="1",
Pages ="16 - 22",
Year = "2026",
Authors ="Dr. Ranga Rao Velamala"}