Inside the machine, and the thoughts beyond it

Right now the only way to see what KaryoSpace actually stores in MongoDB is a third-party tool. I want to fix that with a native data browser, a schema map, and a live view of how a question becomes an answer.

What if there was just a backlog where you type the feature you want, a loop builds, tests, and ships it, and you reply on the same line if it's not right yet? The parts that already work, and the parts I haven't solved.

The standard advice is to filter LLM outputs. I filter inputs instead, intercepting before the model is ever invoked. Here's the design, the reasoning, and what the adversarial tests revealed.

Most AI products are black boxes twice over. The model is a black box to the developer, and the product is a black box to the user. AIO changed that, and what it surfaced was not what I expected.

The queries that return no sources are the most valuable signals in any AI assistant. Here is how KaryoSpace turns retrieval failures into a structured gap-closure loop.

Every time a user opens KaryoSpace, a background pipeline assembles a personalized feed from incidents, emails, calendar, and messages. Here is how the synthesis layer works and why the cache TTL matters.

Model Context Protocol promised plug-and-play tool use. Reality involved subprocess lifecycle management, interleaved JSON-RPC notifications, and a concurrency model nobody documents. Here is what I found.

Multi-stage Dockerfile, non-root user, read-only filesystem, cap_drop ALL, and an install.sh that generates secrets on first run and never overwrites them. What it took to make the image safe enough for prod.

768-dimensional embeddings from a local Ollama model, stored as float64 slices in MongoDB, queried with a two-pass hybrid retrieval that combines text search and cosine similarity. No Pinecone, no Weaviate.

No Google Analytics. No Mixpanel. I built the full pipeline in Go: SHA-256 IP hashing, MaxMind GeoLite2, 30-signal bot detection, and an admin dashboard that renders a live world map from raw coordinates.

Every slash command in KaryoSpace is a structured query. After execution, the intent and result pass through storeUserQuery and into the KRE pipeline. Here is the 4-step flow and why commands are better RAG inputs than free text.

No Postfix. No Dovecot. I wrote the full email stack in Go. DKIM signing, SPF verification, STARTTLS, outbound queue with exponential backoff. Here is what broke and what I learned.

coturn TURN server with HMAC-SHA256 short-lived credentials, mesh peer connections for small groups, replaceTrack screen share, and a sync.Once race that only appeared when two participants ended the call simultaneously.

workspace-mcp exposes search, email send, calendar query, task create, and knowledge lookup over both stdio and HTTP+SSE. The transport is detected at startup with isatty, so the same binary works in Claude Desktop and HTTP clients.

Generic OIDC via RFC 8414 discovery, local auth with bcrypt cost 12, RFC 6238 TOTP, org isolation from email domain, 8-hour idle timeout, and a dev login that is triple-gated against production exposure.

A startSprint function that enforces one-active-sprint per project, a validStoryPoints map that rejects non-Fibonacci inputs, a burndown endpoint, and an AI task creator that extracts structured tasks from freeform text via LLM JSON.

Six goroutines query emails, messages, incidents, tasks, knowledge, and notes concurrently. Results merge through a rankResults function that combines MongoDB textScore, recency decay, and per-module priority into a single ranked list.

Signature verification before body parse, idempotency via a billing_processed_events collection, requirePlan middleware with a planCovers order map, and a canAddUser check that enforces free-tier seat limits at the API boundary.

createNotification writes to MongoDB and pushes immediately over WebSocket. Clients that miss the push catch up on the next 60-second poll. VAPID web push handles 410 Gone cleanup. Mark-read filters on user_id to prevent cross-user manipulation.

A standalone seed CLI generates a realistic org: emails, messages, incidents, tasks, and calendar events for five industry verticals. The _seeded safety flag prevents double-seeding. Embedding generation is throttled to one request per second.

Six external systems, one DataSource interface, and a sync worker that handles OAuth refresh, chunked vectorization, and 15-minute polling. Here is the architecture that keeps KaryoSpace's knowledge graph current.

Character-count chunking breaks sentences mid-word. Token-count chunking requires a tokenizer dependency. Word-count with 50-word overlap hits the right balance for a Go RAG pipeline with no ML framework.

No LangChain. No LlamaIndex. No abstraction layer. Just Go, MongoDB, and a pipeline I understand completely. This is what the decision cost me, and what it taught me.

KaryoSpace runs Groq, Gemini, and Ollama behind a single interface. Here is how the routing layer works, what the latency numbers actually are, and when local inference breaks down.

53 commits in November, 438 in March. A $0 infrastructure breakdown, a systemd service that runs Claude as a persistent coding partner, and what happens to institutional memory when the velocity is this high.

Building a real-time messaging system means confronting Go's concurrency model head-on. Here is the Hub design that handles fan-out, the @mention token format, and the three race conditions I did not catch in code review.

Syncing Google Calendar and Outlook without creating duplicate events requires a push guard, read-at-call-time credentials, and careful handling of the invalid_grant error that only appears in production.

The SLA worker runs on startup, not on first breach. Here is the priority matrix, the StartSLAWorker design with a cancellable context, and why fan-out notification to both assignee and reporter matters.

Message-ID, In-Reply-To, and References parsed into a ThreadContainer graph, dummy nodes for missing parents, wouldCycle guards, subject-line normalisation for broken clients, and thread_id stored at ingest so rendering is a single lookup.

Personal notes must never leak into org-wide RAG queries. The NoteStore uses user-scoped MongoDB text indexes, and a nil-to-empty-slice fix that took one production bug to discover and five minutes to fix.

Twenty years in enterprise IT. Twelve tools that do not talk to each other. One AI that could fix it — but only if the data fragmentation problem was solved first. This is the idea behind KaryoSpace and why the market is ready for it now.