Paired Local LLM with Obsidian on Mobile, Eliminating Daily Note-Sorting Headaches

A long-time user of Obsidian on their phone has found a solution to the note-sorting headaches that come with managing a large collection of notes. By pairing a local Large Language Model (LLM) with Obsidian, they’ve been able to streamline their workflow and make better use of their existing setup.

The user had spent years building a second brain in Obsidian on their phone, but as the number of notes grew, so did the complexity of organizing them. At first, Obsidian’s tagging system worked well, keeping thoughts organized for over a year. However, after reaching a few hundred notes, the system started to drift apart.

The user found themselves with two tags covering the same topic and one label attached to three unrelated projects. Checking through their notes took only a couple of minutes but revealed more broken connections than they could write down. Most daily logs were stored as loose text without any category, making it difficult to find related information.

Obsidian’s full-text search would have found specific notes quickly if the user had known what to look for. However, this was exactly the problem – not knowing where to start looking or even that a note existed in the first place. The user needed something that could read their entire vault and tell them what connected without needing to name it first.

This is where Retrieval-Augmented Generation (RAG) comes into play. A RAG app indexes documents, then answers questions by pulling relevant passages and showing where each one came from. Since Obsidian stores every note as a plain Markdown file on the device, any app with file access can read the vault.

The user chose LocalRAG! because it runs entirely on their phone without needing an internet connection or a separate server. The model doesn’t run on a desktop; instead, it’s pointed at the user’s vault folder. This means that every interaction happens locally, making it faster and more private than cloud-based solutions.

LocalRAG! handles 25 file formats, including Markdown, and requires a download of about 3GB for its bundled on-device model. After setup, extraction, indexing, search, and answer generation all happen on the phone without needing an internet connection. The user added their vault through the app’s file picker and asked about a specific story setting.

The app returned three notes related to that place, written across a full year. Each answer included numbered citations that could be tapped for more information. By opening each source note and reading it, the user was able to repair links one by one, adding them back into Obsidian manually.

LocalRAG! offers a free plan with five questions per day, including access to its on-device model. The Standard plan costs $4.99 per month, while Pro is $9.99 and comes with a one-week trial. Alternatively, users can bring their own Anthropic API key for full access without subscription.

The user chose LocalRAG! over other options because it allows tapping citations to open the exact passage behind them. This feature was crucial in repairing links between notes. While the on-device model is small and noticeable, it answers questions about notes quite well – but still falls short of a full assistant for complex tasks.

Paired with Obsidian, LocalRAG! doesn’t organize the vault automatically; instead, it helps users find what they’ve written by asking their notes directly. This approach rewards deliberate work, as the user still needs to read each note and decide whether links belong before adding them manually.