01 The situation
Earning trust in a category that’s losing it.
Vedic astrology AI sits at a tense intersection. The category is dominated by apps that fearmonger about doshas and dashas; the wave of AI astrologers tends to hallucinate planetary positions wholesale. A meaningful share of users come in distress, not curiosity.
The product problem: earn trust through grounded math, demonstrable safety with vulnerable users, and measurable rigor — not feature breadth.
02 The decision
Scope down to scope the proof up.
I scoped the astrology surface area down — going Vedic-scoped and deferring everything that doesn’t make v1 more trustworthy — to reinvest the time in proof.
The easy path was adding dasha or matching to look ambitious. The harder bet: a v1 that earns trust ships fewer features built to a higher bar.
Built to a higher bar.
- Sidereal kundli (Lahiri ayanamsa, whole-sign houses)
- Rashi · bhavas · lagna · nakshatra
- Daily transits (gochar)
- Six layered safety rails
- Three labeled evaluation datasets
Explicitly future-work.
- Vimshottari dasha timelines
- Kundli matching / gun milan
- Divisional charts (navamsa, etc.)
- Yogas & doshas (manglik, sade-sati)
- Panchang · numerology · tarot
03Product & design
Pressure-tested every choice.
Decisions before code
- 8 core decisions locked through a structured design phase — scope, interaction model, BYOK strategy, tool set, guardrails, eval method, definition-of-done.
- Interrogated whether the entire build could be re-shaped as portable skill files before committing to the agent stack.
- Switched from a generic
ChatOpenAI(base_url=…)to provider-specific packages after research showed tool-calling would otherwise break. - Cut-first discipline for
knowledge_lookup, styling, persistence; never-cut for chart accuracy + crisis rail + eval harness.
Docs as instruments
- Three dual-audience documents — PRD, Architecture, TID — layered so a non-technical reader can stop after Part 1.
- Designed
AGENTS.mdas a build constitution: scaffold → walking-skeleton → polish, with hard exit gates between phases. - Single-writer-per-file orchestration that mathematically prevents sub-agents from corrupting shared state.
- Manual human checkpoint after every phase, plus an append-only
DECISIONS.mdlog capturing every deviation.
04 The build
The pieces that earn the safety claims.
Shown the way they actually fit together — agent graph, layered defense, six rails, and the datasets that keep the eval honest.
LangGraph state machine. Crisis and injection short-circuit before any tokens or tools spend.
Each layer is cheap-first / expensive-only-when-needed. Higher layers catch what lower layers miss.
Crisis & injection short-circuit in the router. Medical / legal / financial get a nudge + an output rail. Fatalism is output-only.
Deliberately separated. The tune set is held out from the eval set, so the eval can never grade itself.
05 The evaluation
Safety, made measurable.
Deterministic checks + a different-family LLM judge + a multi-model sweep. Cost, latency, tool-count, and failure-rate all logged per run.
Same harness, same golden set, two different models. The cost-conscious winner emerges from the data, not vibes.
| minimax-m2.5 · chosen | kimi-k2.6 | |
|---|---|---|
| Pass rate | 93% | 90% |
| Attack-success-rate | 0% | 0% |
| Latency · p50 | 10.4 s | 20.3 s |
| Avg tokens / run | 3,407 | 4,938 |
| Judge agreement | 92% | 92% |
When the harness reported failures, I separated a real product gap (a persona-jailbreak case slipping a marriage-timing prediction past the rail) from a scorer bug (a check too strict on the agent’s correct refusal) — rather than tuning either away.
06 Artifacts
The real paper trail.
The actual documents that drove the build — open or download them below.
- Product Requirements (PRD)Dual-audience product spec: scope, the trust-first thesis, interaction model, and the definition-of-done.Open
- ArchitectureThe system map — LangGraph nodes, the tool layer, chart caching, and where each guardrail intercepts.Open
- Technical Implementation (TID)Concrete build instructions with version-sensitive APIs flagged inline.Open
AGENTS.mdThe coding-agent build constitution: phased build, single-writer-per-file orchestration, exit gates + manual checkpoints.OpenDECISIONS.mdAppend-only decision log — every deviation, its trigger, and the trade-off, captured as it happened.Open- Evaluation methodologyHow safety is made measurable: deterministic checks, a different-family LLM judge, and the multi-model sweep.Open
- Testing & evaluationPer-rail attack-success-rate, the scorer-bug vs real-gap distinction, and the unit-test surface.Open
- RequirementsThe original requirements brief the whole build was measured against.Open
- READMERun-it-yourself overview: the agent graph, tools, guardrails, and BYOK model selection.Open
- Backend codebase
graph.py,guardrails.py,semantic_guard.py,tools/(geocode, chart, transits, knowledge), singletonembedder.py, BYOK model factory.Source
✦ Signal
I think about agents as measurable systems — and I pressure-test every recommendation against the real product goal, including my own first instincts.
I scoped the product down to scope the proof up. When the eval reported failures, I separated a real product gap from a scorer bug — instead of tuning either away.
The 93% pass and 0% ASR across six rails weren’t lucky numbers. They were the output of a system designed to be measured.