ADHD Coaching Agent

The problem

Parents of children with attention-deficit/hyperactivity disorder (ADHD) often need a concrete next step for a homework fight or a morning that has fallen apart, and they need it when no coach is around. A chatbot can answer at any hour, but it has to stay a coaching tool. It must not diagnose a child or advise on medication, and a parent who writes in crisis needs emergency resources at once.

I also wanted every piece of advice to trace back to a document I could check, and every prompt or retriever change measured before it shipped.

What I built

I built it on my own, as a LangGraph graph that puts a ReAct (reason and act) agent between two gates. The agent reasons in a loop and decides at each step whether to answer or call a tool. The input gate makes one structured call to Gemini 2.5 Flash that flags crisis and jailbreak attempts. A crisis message gets the 988 Suicide and Crisis Lifeline and 911 right away, and a jailbreak gets a refusal. Neither reaches the agent.

The input gate’s call also rates the message as simple or complex, which decides whether the agent runs on Gemini 2.5 Flash or Pro. Greetings and other plainly harmless messages skip the call when a small local embedding model finds them close to known harmless examples. The output gate makes a second structured call on the draft and swaps in a safe fallback whenever the draft crosses one of the lines above.

Memory has four tiers. SQLite keeps every message along with the family profile and goals, and a rolling summary condenses the conversation every five turns. An extractor adds facts about the child to the profile, and episodes record moments such as a parent reporting how a strategy went. These updates run in the background after the reply.

The knowledge base began as 100 curated documents, each a strategy or a piece of guidance with its source, and I later added 8,907 documents from research and education databases such as PubMed and ERIC. Search combines a semantic ranking from Gemini embeddings with a keyword ranking, then reranks the top ten with a cross-encoder that runs locally.

Evaluation

I built an offline evaluation package next to the agent. It writes its own retrieval test set from the knowledge base. A language model pulls statements out of each document and turns them into questions of six types, and a filter drops any question its source cannot answer or that repeats another. The harmless-message shortcut has its own set of 57 labeled messages, and its test fails if a single crisis or jailbreak message would take it.

For whole conversations, 50 synthetic parent personas drive the agent, and language-model judges score each turn on six dimensions, including empathy and boundary compliance. Pairwise judges compare two versions of the pipeline twice with the order swapped and call a tie when the verdicts disagree. The results below come from the retrieval evaluation.

The first retrieval run, on 36 questions over the 100 curated documents, split by question type. Direct lookups had a mean reciprocal rank (MRR) of 0.93, while reasoning questions, which need an inference the document does not state outright, scored 0.20. A revised rewriting prompt lifted reasoning to 0.60 and dropped the comparative question from 1.00 to 0.33. I took the rewriter out of the agent’s path and let the agent decide when to search again. I then tried ColBERT, a model that compares query and document token by token, as a third ranking in the fusion in place of the cross-encoder.

ColBERT put the right document first for four of the five reasoning questions, against one for the cross-encoder, and lost a little on procedures and multi-part questions. Five questions is a small sample, so ColBERT is a configuration option next to the cross-encoder for now.

Next

Next is a retrieval test on the full corpus of 9,007 documents to choose between ColBERT and the cross-encoder, and judge runs over full persona conversations. I have also moved three of the four memory tiers into a Graphiti knowledge graph, and the runner that compares it with the four-tier version is written and waiting to run.