QMD - Quick Markdown Search

Why It Matters

By keeping indexing and LLM inference on the user’s machine, QMD delivers fast, private knowledge retrieval that can power autonomous AI agents and developer workflows without exposing proprietary data to external services.

Key Takeaways

• Local hybrid search merges BM25, vectors, LLM reranking
• Runs entirely on‑device via node‑llama‑cpp
• MCP server enables direct LLM tool use
• Reciprocal Rank Fusion with top‑rank bonuses improves relevance
• Supports CLI, JSON, and agent‑friendly output formats

Pulse Analysis

The rise of personal knowledge bases has outpaced traditional search tools, prompting developers to seek solutions that blend speed, relevance, and privacy. QMD answers this demand by delivering a fully local search stack that indexes markdown files, meeting notes, and documentation. By leveraging SQLite FTS5 for BM25, SQLite‑vec for dense embeddings, and on‑device LLMs for query expansion and reranking, QMD eliminates the latency and data‑exposure risks associated with cloud‑based services, positioning itself as a compelling alternative for privacy‑first teams.

At the heart of QMD’s performance is a sophisticated hybrid pipeline. User queries are expanded using a lightweight Qwen‑3 model, then processed in parallel across both keyword and vector indexes. Results undergo Reciprocal Rank Fusion with a k‑value of 60, augmented by a top‑rank bonus that preserves exact matches. The top thirty candidates are re‑scored by a dedicated reranker, and a position‑aware blend balances retrieval scores with LLM confidence. This multi‑stage approach yields relevance scores that closely mirror human judgment while maintaining sub‑second response times on modest hardware.

For developers and AI agents, QMD’s CLI and MCP server unlock seamless integration into automated workflows. Commands like `qmd query` produce JSON‑ready results that LLMs can consume directly, enabling autonomous agents to fetch context, summarize documents, or trigger actions based on up‑to‑date knowledge. The open‑source MIT license and straightforward installation via Bun further lower adoption barriers. As enterprises increasingly embed AI into daily operations, tools like QMD that combine on‑device intelligence with robust retrieval capabilities are set to become foundational components of next‑generation knowledge management stacks.