Hey HN,
I built RAG Logger, a lightweight open-source logging tool specifically designed for Retrieval-Augmented Generation (RAG) applications. LangSmith is excellent, but my usage is quite minimal, and I would prefer a locally hosted version that is easy to customize.
Key features:
Detailed step-by-step pipeline tracking
Performance monitoring (embedding, retrieval, LLM generation)
Structured JSON logs with timing and metadata
Zero external dependencies
Easy integration with existing RAG systems
The tool helps debug RAG applications by tracking query understanding, embedding generation, document retrieval, and LLM responses. Each step is timed and logged with relevant metadata.