Logo
All posts

I stopped coding 7 years ago. Now I am back with AI.

In this blog I will document my learnings over the 90 days, and the resultant AI harness from these struggles.

Gaurav Sen/Updated /10 min read

In 2019, I stopped coding. I taught system design on YouTube and ran InterviewReady, while engineers wrote its code.

In August 2026, I started coding again. This time I had Claude Code and Codex.

The model, and the harness I built around it. From the inside out: who it is, what it can do, the world around it, what is true now, and what must not drift.
The model, and the harness I built around it. From the inside out: who it is, what it can do, the world around it, what is true now, and what must not drift.

Coding with them, I faced two major problems:

  1. The agent repeated mistakes I had already corrected.
  2. I had to manage a business while writing code.

In this blog I will document my learnings over the 90 days, and the resultant AI harness from these struggles.

1. 17x lower latency

Have you waited on a page that loads one slow list? Our course catalogue was that list.

Every request to the catalogue counted three numbers across the whole platform. From 28 to 30 September, the catalogue's median response took about 1,520 ms.

When a visitor left before that response, the count was cancelled and the response became an error. Our previous server had kept those three numbers for 30 minutes, but its replacement asked the database every time.

So the numbers became a cached snapshot. The server answers from the last snapshot, and refreshes it in the background with its own 60 second deadline.

The catalogue server answers every request from its snapshot, in 87 ms. Once every 60 seconds, the snapshot is refreshed from the database. Before the snapshot, the visitor waited 1,520 ms.
The catalogue server answers every request from its snapshot, in 87 ms. Once every 60 seconds, the snapshot is refreshed from the database. Before the snapshot, the visitor waited 1,520 ms.

With the snapshot in place, the median fell to 87 ms on 1 October. This week it's 62 ms.

2. 3.5x faster database

A cache fixed the catalogue. What do you do when the database itself is slow?

On 14 October, an analytics tile in our admin timed out. Its query read an 805 MB table from start to end.

So we added an index to that table. The query planner ignored the index, and the query still took 3.6 seconds.

The planner had a reason. Four rows in five were page visits.

Fetching a seventh of the table through an index looked costlier to the planner than scanning it.

Our test had only proved that the index could be used. It didn't prove that production would use the index.

The second index held the query's whole answer, for page visits alone. We rehearsed that index on a local table shaped like production, with 1.07 million rows.

The admin's query reads a B+ tree that holds only page visits, and answers in 1.0 seconds.
The admin's query reads a B+ tree that holds only page visits, and answers in 1.0 seconds.

On production, the planner chose the new index, and the query finished in 1.0 seconds. An index that holds a query's answer for only the rows it needs is called a partial covering index.

3. Agent = Harness + LLM

Did the agent find the right index on its own? No. It wrote the first index, and its own test said the index worked.

Why did the agent repeat mistakes it had just been told about? Because the model forgets, and nothing around it remembered.

Agent = Harness + LLM. LLM is out of our control, we can only choose the model. So now what's next in our control, let's focus on that part.

Model is non-deterministic, but we want deterministic system. Like building TCP over unreliable network.

TCP gets reliable delivery out of a network that drops packets. It numbers each packet, checks it, and resends what was lost.

Anatomy of an Agent. The agent's code calls the LLM, its tools and the store that holds its memory. Everything but the LLM is the harness.
Anatomy of an Agent. The agent's code calls the LLM, its tools and the store that holds its memory. Everything but the LLM is the harness.

The LLM is one of an agent's four parts. The other three are the harness, and that is where I worked.

A harness does the same for a model that TCP does for a network. Rules say what is right, code checks every output, and tests prove the checks catch a mistake.

The code, rules and checks that surround a model and make its output dependable are called a harness.

Does a harness change the result if the model stays the same? On 10 October, we gave Codex 35 real bug tickets twice, with one model.

With a single prompt, it judged 3 tickets correctly. Inside a layered harness, it judged 24 correctly, on about half the tokens.

4. What didn't work

What if a harness's checks pass and the code is still wrong?

Then the checks aren't checking.

Have you or have you not atleast tested as intensely as we test prod?

On ir-server, the migrated code has defects planted into it, one at a time. If its tests still pass with a defect in, the build fails.

The gate started with 37 defects on 29 September. It now holds 2,069.

Planting those defects one after another made one build run for 59 minutes. Dealt across parallel jobs, a pull request's checks now finish in 139 to 179 seconds.

Bug reports go through the same checks. Codex writes a failing test and then the fix.

A separate job then proves the test fails before the fix and passes after it. When all 12 checks of a gate hold, the fix becomes a pull request.

When any check fails, the fix is marked for a human. Nothing merges on its own.

5. What worked

Who decides what merges, then? I do, through instructions that became rules.

Five of them kept coming back, until each became a rule:

  1. One unit at a time, judged by me: I approved the book chapter by chapter, chapters 1 to 8 between 12 and 17 October.
  2. Rules as code: a rule I only wrote down came back broken, and a rule with a check stayed fixed.
  3. Parallel work: every session got its own copy of the repository, called a worktree, and this Mac holds 77 of them.
  4. Impact over activity: a report that listed work, with no result, was sent back.
  5. Heavy work on the cloud: video, training and long builds run on AWS, never on the laptop.

Is what you are doing useful? What is the impact. Tell me with example.

Do all this stuff, in parallel when you can, in sequence when you must.

Run everything on cloud, store it on cloud.

Final Thoughts

Where do those rules live? Each of them lives in a harness, and here are the three harnesses side by side.

Part of the harnessBookReelsServer
What is rightMy rewrites, as rulesMy cuts, as exemplarsProduction behaviour
What checks it174 checks65 coded rulesTests and planted defects
Who judgesI approve each chapterI accept each proxyA 12-check gate, then me

The model writes the code, and the harness decides what is accepted.

Summary

Here are some stats on my coding contributions (treat them like heuristics, not KPIs).

WhatCount
Pull requests merged1,590
Commits on default branches6,398
Days in a row with a commit66
Repositories touched26

The largest share of those pull requests, 546 of them, went into one new Go server, ir-server. It took over the old InterviewReady server's routes. The old server was stopped on 4 October.

The new server's two biggest wins were about speed. A page's slowest read got 17x faster, and an analytics query got 3.5x faster.

WhatsApp