Engineering

Graph RAG for enterprise documents: the graph is the easy part

Extracting entities and relations into a graph is the cheap step. In an enterprise, Graph RAG succeeds or fails on source authority, entity resolution, permissions, updates and evaluation. Here is how we build it.

Karthik Kamalakannan ·

LLM evaluation and reward hacking: how AI agents game evals, and how to build ones that hold up

A reference page on LLM evaluation and its main failure mode, reward hacking. Definitions, documented examples, how to detect gaming, how to design evals that survive production, and a checklist for enterprise teams.

Karthik Kamalakannan ·

We now have a CI mesh with our laptops

Two weeks ago our CI was one spare MacBook Air called CASE. Now it is two office Airs, CASE and TARS, plus a mesh of team laptops that only take jobs when it is safe. How the routing works, how we keep deploys off laptops, and the five things that broke along the way.

Karthik Kamalakannan ·

We moved our own error reporting to OpenFault

FeatureOS and SupportWire now send their errors to OpenFault, the self-hosted error tracker we build. Here is what we changed in each product, the crash it caught, and what is still rough.

Karthik Kamalakannan ·

We turned a spare MacBook Air into our CI

Our CI bill tripled after we added tests to every repo. So we moved our GitHub Actions runners onto an 8 GB M1 MacBook Air in the office. What it cost, what got faster, what got slower, and how we kept it safe.

Karthik Kamalakannan ·

FeatureOS is now real-time everywhere, all at once

Our real-time layer looked fine from the browser, but the server barely sent anything. What we found, what we fixed, and the bugs it exposed.

Karthik Kamalakannan ·

Introducing OpenFault

A self-hosted, Sentry-compatible error tracker that runs in one container with SQLite, and talks to your AI agents over MCP. Open source, MIT licensed.

Karthik Kamalakannan ·

Duplicate detection for feature requests: calibrating an embedding threshold against human merges

On our own FeatureOS board, a cheap embedding model puts the right duplicate in its top five suggestions 76% of the time, but once a wrong merge costs three times a missed one, the best auto-merge threshold catches 15 of 101.

Karthik Kamalakannan ·

Where we are on the road to AI-native engineering

Meta's write-up on building Forum with agents made me take an honest look at our own engineering. Here is where Skcript is today, the stages ahead, and why we move carefully.

Karthik Kamalakannan ·

Older posts