A Local AI Search Engine for My Notes

I take notes fast and title them badly. Files named stuff.md and todo2.md pile up, and three weeks later I’m in the field trying to remember which one had the valve cover torque spec or the switch port I’m not supposed to move. Obsidian wanted $4 a month to sync those notes to my phone. I already run a homelab with a GPU in it, so I built the sync and the search myself.

Design: three jobs, one worth coding

The whole design falls out of one fact: an Obsidian vault is just a folder of markdown files. That splits the problem into three separate jobs. Editing is Obsidian itself, free on both PC and phone. Syncing is Syncthing, peer to peer over my LAN with no cloud account (on Android use Syncthing-Fork from F-Droid, the original app is discontinued). Neither of those needed a line of code from me, and writing my own sync would have been months of work to end up with a worse Syncthing. The only layer worth building was the AI on top.

That layer is three small Python scripts, standard library plus the chromadb client, talking to the LM Studio server and ChromaDB container I already run for my other AI projects.

The digest agent

First script walks the vault and fingerprints every note by mtime and size. Anything new or changed gets summarized by the local qwen3-8b model and appended to a daily digest note inside the vault:

full_path = os.path.join(root, file)
rel_path = os.path.relpath(full_path, vault_dir).replace(os.sep, '/')
stat = os.stat(full_path)
fingerprint = f"{stat.st_mtime}:{stat.st_size}"
if rel_path not in state or state[rel_path] != fingerprint:
    changed.append(rel_path)

The digest lands in _agent/digest-2026-07-12.md, and since that’s just another markdown file in the vault, Syncthing carries the summaries back to my phone with zero extra plumbing. Anything starting with an underscore or a dot is excluded from scanning, so the agent never summarizes its own digests or Obsidian’s config folder.

The index

Second script chunks each note to about 1500 characters on paragraph boundaries, embeds the chunks with the nomic-embed-text model through LM Studio’s OpenAI-compatible endpoint, and upserts them into a collection on my existing ChromaDB server. Edits are handled by deleting a note’s old chunks before upserting the new ones, and notes deleted from the vault get their chunks removed on the next run.

docs = [f"[{title}]\n{c}" for c in chunks]
embeddings = embed_texts(docs, base_url, embed_model)
collection.delete(where={"path": rel})
collection.upsert(
    ids=[f"{rel}::{i}" for i in range(len(docs))],
    embeddings=embeddings,
    documents=docs,
    metadatas=[{"path": rel} for _ in docs],
)

Prepending the filename to each chunk costs nothing and helps retrieval when the title actually means something.

The search server

Third script is a stdlib ThreadingHTTPServer on port 8090 with two endpoints. POST /search embeds the question and returns the top 5 matching chunks in milliseconds, pure vector math, no LLM involved. POST /answer hands those same chunks to qwen with a strict prompt:

ANSWER_SYSTEM = (
    "You answer questions using ONLY the provided notes. Cite the note path "
    "in parentheses after each fact you use. If the notes do not contain the "
    "answer, say so plainly. Be concise: a few sentences."
)

The page shows the raw matches instantly and the answer fills in 10 to 30 seconds later on my RX 6600. That sounds slow until you compare it to the alternative, which is scrolling through my own bad titles or fighting a search engine. For pulling up half-remembered info, the wait is fine.

Does it actually find things

I seeded the vault with deliberately awful notes and asked vague questions. “How tight do the valve cover bolts go” pulled stuff.md at rank one. “Which switch port am I not allowed to touch” pulled the right homelab note with a wide margin over the noise.

It also has the benefit of being a simple keyword search for the AI summarized notes, none of which reach the internet. That portion is instant, and the additional AI contextual searching does the rest in short order.

Reaching it from the field

I already run WireGuard on OPNsense, so remote access was one firewall rule on the PC hosting the server:

New-NetFirewallRule -DisplayName "vault-search" -Direction Inbound -Protocol TCP -LocalPort 8090 -Action Allow

Phone connects to the tunnel, browser hits the LAN address on port 8090, and the page is bookmarked to the home screen so it opens like an app. Nothing about my notes ever touches a cloud service.

A build gotcha worth writing down

I had my local model write the first drafts of these scripts through opencode, and the dispatch kept hanging with no error, no output, and nothing in the LM Studio log. Turns out opencode silently blocks waiting on stdin when the prompt argument contains double quotes. Short unquoted prompts worked, which made it look random. The fix is to feed the prompt through stdin instead of an argument:

tr -d '\r' < spec.txt | opencode run --agent codegen > out.py

Four timed-out runs before I found that. If your opencode prompts hang only when they contain quotes, that’s why.

Running cost is zero dollars a month. The models were already loaded for other projects, the ChromaDB container was already up, and the sync is peer to peer. Next step is moving the scripts onto the server with the rest of the lab and putting the digest and index runs on a schedule, but everything above works today from my pocket.