Atlas

Hey — Nishil here.

AI writes more of our code every week, and it makes a particular kind of mistake: it edits one spot and quietly breaks something connected that it never looked at. Types and tests often miss that, because the broken thing wasn’t what you were checking.

So I built Atlas — a deterministic verification engine. Change code, and it figures out which parts of the running app the change reaches, checks those for real (HTTP + a headless browser), and hands back a verdict: clean, changed, or broken. No AI opinion in the loop.

The engine got finished. Then I did the honest thing and measured it. Same bugs, three setups: no checks, the usual checks, and the usual checks plus Atlas. What I cared about was the fake fix — everything green while the bug is still there.

On the bugs where the AI was “fixing” something in one place and silently wrecking a shared feed somewhere else, Atlas actually helped. On the ones where normal checks already caught the problem, it added nothing. The score that mattered wasn’t the win rate. It was what the score was made of.

Catching breaks after the fact was promising as a heads-up and not worth it as a product. The part that knew two routes shared a function — the dependency graph — was the real asset. Prevention before the edit beats grading after.

So Atlas, the verification engine, stops here. The repo is the record of that journey. What I want to build next is different: an architectural, design-principle linter for AI-generated code — hygiene on every change, so the codebase stays well-designed by construction, not cleaned up later.

If that story resonates, the write-up and the engine live on GitHub. Stars help. Issues and PRs do too.