How to train a Transformer (as a software engineer with no ML experience)

This November, I entered CodinGame's AI bot challenge, an eight-day contest with participants from around the World.^[1] Claude agents wrote all of my code, while I set the harness around it.

I noticed that engineers from Amadeus did quite well here, which helps create aviation planning software. Are optimisation problems common in flight planning? Who knows. 🤷

As with any AI contest, the best submission wins. To do this, I needed my bot to look beyond simple algorithms: I needed it to train a neural network designed specifically for the contest.

Since I had never trained a neural network, I kept simple goals:

1. The network had to understand a game board.
2. It had to run inside 50 ms, in one source file of at most 100,000 characters.
3. A network that looked better in training had to play better in real games.

In this blog we will see how I trained a transformer for it, step by step, and what each experiment did for my goals.

1. Decide what the network does

Where does a network fit in a bot that searches? At the end, where the bot has to choose.

Each turn, the search improved one plan for all our units, with about 14,000 edits in a mid-game turn. The judge then built a table of our candidate plans against the opponent's likely replies.

Each cell of that table was played forward one turn and scored by the network, about 64 times in a mid-game turn. So the network's job was narrow: look at a position, and say how good it is for us.

2. Convert application state to tokens

How does a transformer read an application's state? The same way it reads text: as a list of tokens.^[2] In our bot, the application state was the game board.

Attention lets each token look at every other token and weigh which ones matter.

Our teacher network had one token for every house, one for every triangle, and one for the whole board. Each house token carried its units, its owner and its position. Each triangle token carried its owner and whether either side could capture it.

Attention also got a learned bias for how two tokens relate. Two houses one step apart relate differently from two houses four steps apart. A house that is a corner of a triangle is marked as one.

With those tokens and relations, the network saw every board from the side about to move. It answered two questions: how likely is this side to win, and by what margin?

That's the one design decision I needed to understand. The agents wrote the model, its training loop and its tests.

3. Train it to meet application-specific goals

Where does a network learn to judge, when nobody labels positions for you? AlphaGo answered that in three steps.^[3]

1. Supervised learning: learn the moves and results of strong players from their games.
2. Reinforcement learning: play against yourself, and retrain on those games.
3. Distillation: a large, slow teacher labels positions, and a small student learns to copy it.

Distillation came first for me. On 4 November, a small network learned to score positions the way a slower, stronger search did. Once the judge also saw the opponent's own plans, that bot beat the previous one 177-79 and placed 34.65, my best score in Silver.

Self-play came next, because I asked for it. On 5 November the bot played itself on a farm of AWS machines and my graphics card. On a rating scale with the previous bot at 1500, the self-play bots climbed to 1758.

Online, the first of them placed 17.54, at 198 of 426 in Gold. The bot it replaced had placed 16.77, at 172: a higher score, but a worse rank. Better against itself didn't mean better against real players.

So on 7 November the networks learned from 1,918 games of top players instead. That bundle beat the previous build 248-120-32 and raised my placement by 1.20 points. It also carried a lead guard and a timeout fix, so the network can't take all the credit.

4. Make it fit in 50 ms

Can a network ship inside a single source file? Ours had to, in at most 100,000 characters.

The live student had about 99,000 parameters, so it was trained to keep its accuracy at 7 bits per weight. Its weights were packed into a string with an entropy coder, and took about 18,010 characters.

Size still ended ideas. A variant with a second embedding packaged to 102,999 characters, and was cut.

So the student had to get smaller. On the last day, a 42-minute run distilled it into five smaller networks at once.

The run drew from 1.47 million positions, swapping the players and reflecting the maps. It held out 14,111 real positions for testing.

The pick, d24L2, had 9,515 parameters and ran about nine times faster. On the held-out positions it chose its teacher's top move 79.8% of the time. Its value agreed with the teacher's at a correlation of 0.990.

5. Test it inside the bot

Does a network that's better in training make the bot better? Not always, as d24L2 showed.

With 96 games sharing the test machine, the bot with d24L2 and a deeper search won 64.5% against the live bot. With 48 games, one per core, it won 55.7%. It also failed my opening tests, and online it placed 25.69.

Reinforcement learning did worse than d24L2 inside the bot. One overnight policy beat its own parent 819-203 when both played by policy alone, and scored 46.0% inside the search.

A policy that was better on its own made the searched bot worse.

So judge a network inside the system it serves, under that system's clock and load. Its own training numbers aren't the test.

6. Weigh the network against the rest

How much did the transformer move my goal, the placement? Less than I hoped. Of 11 network and reinforcement-learning ideas, 5 passed a test against the live bot, and 1 raised the placement.

The placement's biggest gain of the contest came from structure. The bot planned against the opponent's best reply to its own plan, then re-planned.

At the same compute, that rule won 67.6%. Ten times the search won only 54.8%.

The network was one part of the bot. The decision rule around it did more for the placement than the network, or ten times the search.

Summary

1. A transformer reads a list of tokens, and attention lets each token weigh every other token.
2. Application state becomes tokens. A game board became one token for each house, one for each triangle and one for the whole board.
3. AlphaGo's recipe trains a network in three steps: learning from strong players' games, learning from self-play, and distillation.
4. Distillation trains a small student to copy a large teacher. The student d24L2 picked its teacher's top move on 79.8% of held-out positions, and ran nine times faster.
5. A network trained for 7-bit weights fits inside one source file.
6. A network is tested inside the system it serves. A policy that beat its parent 819-203 alone scored 46.0% inside the search.

Sources

1. Fall Challenge 2026, CodinGame.
2. Attention Is All You Need, Ashish Vaswani and others, 2017.
3. Mastering the game of Go with deep neural networks and tree search, David Silver and others, Nature, 2016.