The final specification

Reading this page

This specification describes the expert bot and explains why we chose its methods. It's written for a reader who knows data structures, algorithms and the basics of neural networks, but hasn't seen how a game-playing system like this is put together.

Boxed asides explain a concept the first time it appears, and the concept index lists them all. Figures captioned "textbook" illustrate a general method, while implementation figures identify their source.

Objective

The objective is first prize in the UNSW Battlecode Loong competition, whose rules are in the competition documentation. Under the Slay Queen rules from 1 October, a game is won by eliminating the other team. Otherwise, after round 500, the queens (dragons 0 and 1) are compared, then the longest living dragon, then total living length.

The expert line (bots/expert) is the only bot line in development, and its bot is judged by reviewing its decisions in replays in just viewer. Win rates, ratings and other aggregate statistics select games for review and tune parameters, but they don't establish that a decision was sound.

The game in brief

Two teams play on a rectangular grid, 10 to 64 cells a side, that wraps at both edges like a torus. Each team controls dragons: snakes whose head moves one cell north, east, south or west each turn while the body follows. Eating a pearl adds a segment. Crossing kelp, or running into your own body or another dragon's, kills the mover, and two heads meeting kill both. Portals join pairs of edges.

A dragon can also sprint several steps in one action, where the first ceil(L/4) steps are free for a dragon of length L and each later step costs a tail segment. It can split, handing its rear k segments to a new dragon, which starts as a fresh program with no memory. There's no way to stand still. A team may have up to 64 dragons alive, and they all act one after another in ID order every round.

Each dragon sees only the 7×7 square around its head and runs in a separate process with no shared memory, so anything beyond that square has to come from memory, inference or teammates. Teammates communicate only by sonar: up to four 64-bit values a turn, one in each direction, each travelling in a straight line until it hits kelp or the first dragon segment. The receiver doesn't learn who sent it, and the enemy can receive it too. On its next turn the sender gets echoes, counts of what its rays hit (kelp, allied body, allied head, enemy body, enemy head). When a dragon dies, alternate segments starting at its head become pearls, ceil(L/2) of them, so a death feeds whoever is nearby.

Every dragon's turn is metered. The judge, the organisers' program that runs matches, compiles each bot to WebAssembly and charges a fixed number of "points" for every instruction executed. Each dragon gets 100 million points a turn, 48 MB of memory and one thread, and a team's whole submission is at most 4 MB. A dragon that exceeds its points dies.

Teams upload bots to the ladder, the competition's online matchmaking, where ranked games move each team's Elo rating (Elo, 1978). The difference between two teams' ratings predicts how often one beats the other, so a 200-point gap means the stronger side wins about three games in four. The tournaments are played on unseen maps.

The design

The design is expert iteration (Anthony, Tian and Barber, 2017), the scheme behind AlphaGo Zero and AlphaZero (Silver et al., 2017 and 2018), applied to a decentralised team by centralised training with decentralised execution, CTDE (Lowe et al., 2017; Yu et al., 2022). Expensive, fully informed reasoning runs offline, where compute is cheap, to improve the training targets for a small network. During play, each dragon runs that network on its own observations and searches with the rest of its turn's budget.

Related systems

AlphaGo combined imitation of human play, self-play and tree search. AlphaGo Zero and AlphaZero kept the search and dropped everything else, which is expert iteration in its purest form. AlphaStar and OpenAI Five learned by reinforcement learning in team and hidden-information games, without search at play. This design takes AlphaStar's start from imitation, AlphaZero's search-generated targets and OpenAI Five's one network shared by many units, then adds what Loong needs: a belief for the hidden board, a privileged teacher, and search within each dragon's own budget.

Two measured limits shaped this. Simulating the whole team faithfully for every decision doesn't fit in a dragon's turn, so decision quality has to come from what the network learns offline. And hand-weighted scores, which don't check the consequences of a choice, plateau, so the learned value and the search together replace them.

Reading the loop clockwise, games are played against a league of opponents (part 6):

Opponent	Description
Foil	Our earlier hand-written bot and our strongest, in its version foil-0039.
Pool	Frozen copies of earlier bots, compiled once and stored as registered builds so each plays identically every time.
Mimics	Networks trained to play like particular top ladder teams.
Current network	The network plays against itself.

The teacher (part 5) replays positions from those games with a full view of the board and searches for better moves. The network is trained on them, exported as integer code for the judge (part 2), and built into the bot, which joins the league.

Within one turn, the dragon updates what it believes from what it sees and hears (part 1). Its options (part 3) propose a short list of candidates, concrete actions for this turn such as "head for that pearl" or "escape east". The network (part 2) gives each candidate a prior, its estimated probability of being the best choice, and estimates the position's value. Search (part 4) checks the most promising candidates by simulating ahead, and a route controller turns the chosen candidate into this turn's move and sonar.

Principles for every component

Every component is optimised towards its floor before the parts are balanced against each other, and its measured cost is reported against two lower bounds. Its algorithm's floor is its current algorithm's lowest possible cost in judge points, with each operation at its cheapest legal WebAssembly sequence. Its operation's bound is the lowest cost of any algorithm computing the same thing. For the network, the judge's prices allow at most about 3.2 multiply-adds a point (Budget).

Every constant in the bot, the teacher and the trainer (caps, radii, counts, exploration constants, expiries and reserves) is set by a derivation, a bound or a decision-theoretic criterion wherever theory gives one. It's measured only where theory gives none, or to confirm a rule: the bot's constants at equal cost in the judge, and the offline stages' by strength gained per GPU-second and per dollar. A constant set by neither is a placeholder.

Each part's method is a choice among known methods. Once the current design is implemented, optimised to its floors and playing on the ladder, so that its rating is the baseline, each of the seven parts gets a decision record covering:

Entry	Required content
Known methods	The best known methods for the problem, including recent work and other teams' approaches.
Chosen method	The method and the theory or measurement behind it.
Credible contenders	An equal-cost test against the current method, or a stated reason for rejecting each contender.

A contender that wins replaces the method, after a reviewed specification change. The records are revisited where the evidence points: information gaps in the value-lost report (part 5, which sorts every departure from the teacher's choice into one of three causes) point to the belief and sonar, ranking gaps to the network, candidate gaps to the options, and a search that loses to its own network to the search.

The seven parts

1. Belief state per dragon

Each dragon keeps a hand-built Bayes-filter world model (Thrun, Burgard and Fox, Probabilistic Robotics) of the map, pearl timers, enemy sightings and decoded sonar, and encodes it as a stack of input planes for the network. The filters do the inference explicitly, so the network has less to learn and can be smaller.

The remembered map and the filters

The remembered map is the observation layer: every cell's latest sighting, by the dragon or a teammate, with its round. Over it, every other dragon has a Bernoulli filter for its chance of being alive and a histogram filter over its head's cell and heading (Ristic et al., 2013; Thrun, Burgard and Fox, section 4.1). Lengths are exact where a whole body or a status shows them, a status being a dragon's own report about itself over sonar. Each filter also keeps its dragon's body as last seen. The segment i places behind the head stays put for length − 1 − i of the dragon's moves, so a sampled state joins the drawn head to that body by a shortest way through free cells and keeps the seen segments in place, instead of laying the body straight. A queen's filter is never dropped, and not seeing her moves her without lowering her chance of being alive. Each spawning tile has a renewal process for its next attempt (Feller), and each cell's pearl a two-state hidden Markov model.

Sonar

The teacher sees the whole board and needs no sonar, so sonar's content is chosen by how much of the teacher's advantage it recovers for the dragons in play (part 5). Other teams can decode our broadcasts as we decode theirs, so they're encrypted with a key for each team, derived from a secret kept out of source control. In self-play neither side reads the other's sonar either, and a message carries only what its sender knows while delivery uses the true state.

Each ray carries one frame of several variable-length records chosen for the ray's likely reader, with check bits that reject the other team's values and noise. The number of check bits minimises expected loss: the chance a forged frame is accepted times that forgery's cost, plus the bits times their value if they carried facts instead. A received fact enters the filters as an observation, so its confidence decays by prediction. Relays keep their facts' original rounds, newer facts replace older ones, sight beats report, and a dragon ignores its own reports when they come back. A teammate's intention record (its chosen option and where it's heading, part 3) becomes its latest plan in the belief, so the crop marks its target, a candidate aimed at the same target says so, and the search's samples move that teammate toward it.

Map symmetry

Every map is a left–right mirror, a top–bottom mirror or a half turn, and bots aren't told which. The belief rules a symmetry out when partner edges differ, or when partner spawns can't share a countdown (one spawns and the other doesn't, or an attempt still to come differs at both sightings), using its own sight and teammates' relayed map facts. A forged value can occasionally pass the check bits, so a teammate's report needs confirmation before it rules a symmetry out. Until then a contradicting report only lowers that symmetry's weight, and the next report of the same fact replaces it. Once one symmetry is settled, a reported edge or spawn that differs from its settled partner is refused, because a forger doesn't know the map.

The map never changes, so a symmetry ruled out stays out, and each turn only that turn's new facts are checked. The belief fills each unseen edge, spawn and countdown wherever every remaining symmetry agrees, marked as inferred. The enemy queen starts at our queen's start mirrored by the true symmetry, so the belief marks one cell for each symmetry that still fits, at most three, for the network, the search's samples and the queen strike (an option that attacks the enemy queen, part 3). Current positions are never inferred by symmetry.

Implementation and filter size

The belief is one integer implementation in the shared library, run identically by the bot and the GPU engine, our own CUDA simulator of the game, which plays thousands of games side by side for training, each in its own lane of the batch. Each filter's size is set adaptively within the turn's budget by a bound on its approximation error, the KL bound of KLD-sampling (Fox, 2003) or its effective sample size (its weights' worth in equally weighted states), instead of fixed caps on states, on the dragons tracked or on a drop threshold. Equal-cost comparisons set the bound.

Belief calibration and informativeness

Identical bytes from the bot and the GPU engine only prove the two agree, so the belief is also checked against the true state of recorded and self-play games. Some checks hold whatever the opponents do and are settled once: no impossible state, certainty wherever the rules determine it, and coverage, meaning the truth stays inside the filter's support (the states with any probability), and a motion model that never gives a legal move zero probability guarantees it except where caps drop states. Calibration and informativeness depend on how the opponents move, so they're measured every training cycle and again whenever fresh ladder games arrive.

2. The policy and value network

The network reads the belief planes of part 1 and the candidates of part 3, and returns a score for each candidate and a value for the position. The judge prices every instruction at a fixed cost, so an evaluation's cost is an exact function of the network's shape, and the rest of the turn is sized from it (Budget). It runs in exact integer arithmetic, int8 weights times 7-bit activations, sixteen multiply-adds an instruction, with its weights compiled into the code (below), so the bot reproduces the trained network bit for bit.

The network is feed-forward, with memory in the belief instead of a recurrent layer. Its shape is chosen by playing candidates of equal measured cost against each other in the training loop, recurrent ones included. Until that comparison runs, the bot's network is a placeholder shape from the kickstart (part 5): about 0.42M weights in a 3 × 3 stem and four 3 × 3 trunk layers of 32 channels. An evaluation is most of a search simulation's cost, so the network's shape sets how many simulations a turn affords.

Beside the crop, the network reads a 15 × 15 overview of the whole map, a vector of scalars and a 48-byte record for each candidate. A candidate's record includes the room left after its action: the cells reachable from its end, at most 32, with the dragon's own segments admitted once they've left, and with cells where another dragon was seen this round, or a first step beside another dragon's head seen this round, shut.

The body summarises the trunk, the overview and the scalars into a global vector, which the value head turns into one number. The policy head runs once for each candidate on the trunk's features at the cell where that candidate ends, the candidate's own bytes and a context from the global vector, and a softmax over the candidates' scores gives the policy.

Four additions to the inputs, all drawn from what the belief already holds, remain to be built:

• enemy tokens: one for each tracked enemy, summarising its filter (its likely positions and their spread, its length, the time since it was seen, and its reach to our head and to each candidate's route), read by a small set encoder or attention
• route features for each candidate: the belief-weighted danger along its route, its arrival margin against the nearest enemy and the share of unknown cells on the way
• spatial features for candidates far from the head, taken from the overview or a finer map
• a finer whole-map layer than the overview

The value head reads the enemy tokens' lengths among its inputs. Each addition stays only if it wins against the current inputs at equal measured cost.

Integer arithmetic in the judge

The weights are immediates, constants written into the instruction stream itself, so loading one costs no memory access or address arithmetic. That's why the network is compiled into the bot as code instead of loaded as data (Budget).

The turn-stable crop and the incremental stem

The same idea works here if the inputs mostly don't change between evaluations, so the crop's 56 channels are arranged to keep the stem's work across the search's leaves and the dragon's turns. The crop is 17 × 17 real cells around the head in the dragon's frame, so the stem's 15 × 15 outputs see true neighbours instead of a zero border. Channels 0–47 (map memory, edges, dragons, marks, and inferred and told facts) depend only on the cell, the view and the facing. Times in them count from an epoch base 64 to 127 rounds back, so a remembered cell keeps its bytes. Channel 48 holds the round past that base, and the network derives each age from it. Channels 49–55 hold what moves every round, such as the filters' head chances and the steps from the head, so they're read at the stem's centre tap only.

The bot caches each stem output's int32 sum over channels 0–47 and reuses it until a byte in its window changes. Integer sums don't depend on their order, so the outputs equal the full stem's bit for bit. Evaluating several leaves together saves almost nothing, by contrast, because every relaxed dot already reads its weight from a local.

Points an evaluation on help.map, seed 7

Layer	Points
Full stem	1.109M
Incremental stem	666K
One trunk layer	666K

3. Options as the action space

Behaviour modules such as harvest, escape, door clearance and the queen strike generate candidate targets and joint plays, and the network scores those candidates instead of raw moves. In the options framework (Sutton, Precup and Singh, 1999), a chosen option is executed by a route controller and ends when its termination condition holds.

Joint plays follow STP's plays (Browning et al., 2005): a few explicit cooperative tactics, such as guarding the queen or clearing its exit, coordinated over sonar. A dragon tells teammates its option in an intention record (option code, target and expiry), and a joint play is a one-record offer, and the recipient accepts it by acting on it, as its next status shows. There's no waiting action, so holding loops are candidates, and fatal actions stay candidates.

The five joint plays are short sequences between two dragons: guarded escape (the queen holds, a worker clears the threat, the queen sees a usable exit and leaves), door clearance (the blocker releases, the beneficiary sees the gap and passes), feed transfer (a donor deposits, the beneficiary collects), vanguard handover (a replacement arrives, then the incumbent leaves) and survey recruitment (a parent splits, the child takes a lane and surveys). An offer that isn't accepted leaves both dragons free to do something else.

Generators are drawn from the behaviour catalogue, and they're added in measured order, because each costs candidates and points. The first are harvest timed to spawns with holding loops, escape, the queen strike, feed and recombination (its timing learned), the rearguard, and the child map upload with the split that puts the child behind its parent. Which generator to use, and when, is the network's choice.

Generators are added, rewritten and removed by measurement (part 5). One is added when candidate-gap losses cluster on a situation with no generator, rewritten when its candidates are chosen but the teacher keeps preferring another variant of the same behaviour (another target or route), and removed when it's rarely chosen or its ablation (the bot without it, measured against the bot with it) shows no gain at equal cost. When ranking gaps dominate, the network is trained and the generators are left alone.

Queen and champion options

Four generators protect our queen or feed the team's longest dragon, offering their candidates to the same network and search:

Every move and sprint candidate also carries a survival depth: the longest certainly survivable continuation, up to eight further single steps. A bounded depth-first search finds it, respecting blocked edges and occupied cells, letting the tail move, and growing the body for remembered pearls. Each search examines at most 32 edges and all survival work in a turn shares a 4M-point cutoff, so when the budget runs out the depth is a lower bound. Fatal actions, splits and unknown endpoints have depth zero.

4. Online search

The search is determinised Monte Carlo tree search in the form of information-set MCTS (Cowling, Powley and Whitehouse, 2012). Hidden enemies are sampled from the belief, and the network supplies priors and leaf values (Silver et al., 2017).

The leaf values come from the value head distilled in part 5, read only once it passes part 5's checks. The root simulates its likeliest candidates and narrows them by sequential halving, which improves the policy even at a few evaluations per turn (Danihelka et al., 2022). In play and in every evaluation of a bot, teacher or network, the Gumbel draws are zero, so sequential halving chooses deterministically by prior and value, as in Gumbel MuZero's evaluation. The teacher uses the draws to explore during self-play data collection (part 5).

The number of simulations is set by what a node costs, so the encoding, the belief's prediction and the simulator are optimised to their floors before the state cap, the simulations, the depth and the network's size are balanced against each other:

One search simulation on help.map, seed 7 (Budget)

Work	Points
Leaf's network evaluation	4.24M
Leaf's encoding	1.15M to 1.26M
Complete simulation	5.4M to 5.9M
Simulations a turn	about 5.7

Each dragon's 100M points are its own and lapse when its turn ends, so every dragon searches with its own budget, quiet workers included, and stops by the judge's virtual clock.

Other dragons in the search

Other dragons in a simulation are never frozen, and never act on the searcher's sampled hidden state or its knowledge:

Dragons	Model
Nearby allies	The route controller follows their reported intentions.
Nearby enemies	Chance nodes over safe moves, with priors from one network evaluation each at the root, or from their team's mimic (part 6) once it passes a check of how faithfully it plays like that team.
Distant dragons	They keep their heading or intention.
Newborns	They follow a fixed rule.

The distance that makes a dragon nearby, the fixed rules for distant dragons and newborns, and limiting enemies to safe moves are placeholders, each kept only if it beats a richer model at equal cost. Results are wins, draws and losses under the Objective's exact scoring order, without survival shaping (extra reward for staying alive, which would discourage useful sacrifices). A complete action is ready before the search starts, and search memory fits the 48 MB in dense arrays.

Once the search reaches a useful depth, a transposition table will be tested. It hashes positions by Zobrist keys over the dragon's observation-relevant state (Zobrist, 1970), XORing a random 64-bit number for each feature present, so a move updates the key cheaply and move orders that reach the same position share their statistics. The table stays only if it saves points per useful node or wins at equal cost.

Alternatives for hidden information

Information-set MCTS is one of several ways to search when a player can't see the whole state. They differ in what the tree's nodes stand for, and so in what they cost and what they can guarantee, and every one besides ours is a contender for part 4's decision record (Principles). Playing the network's top candidate with no search is the baseline. Search has to beat it at equal cost, so the comparison is rerun whenever the network or the search changes, because a stronger network narrows the gap.

Search across turns

A dragon's process lives for its whole life, so its search tree persists between its turns, as AlphaZero reuses its tree. On the dragon's next turn the root moves to the node reached by its last action and the observation that followed, and every other branch is freed. If no grown node matches that observation, the search starts from a fresh root. The kept statistics were gathered under the earlier belief, so their visit counts are discounted by a factor derived from how far the belief moved, such as the KL divergence between the old and new root beliefs. Nothing is pruned for good: each turn the root regenerates its candidates from the new belief, and progressive widening admits more moves at a node as its visits grow, so a move left unexplored earlier is grown whenever the search comes to favour it.

The Gumbel root's c_visit and c_scale (Danihelka et al.) and PUCT's constant (AlphaZero) are re-derived for our value scale and simulation count, progressive widening's exponent is fitted in Couëtoux et al.'s form, and the remaining constants are tuned jointly at equal cost by SPSA or CLOP. Nodes live in a fixed arena within the 48 MB, and when it's full the search recycles its least-visited leaves.

Spending the whole budget

The judge's virtual clock advances 1 ns per point, so a dragon always knows its exact remaining budget, and every turn spends its full 100M points less a reserve for its reply. The network and the root come first, then growing the reused tree, then an all-pairs distance table over the dragon's known map, 16 MB on a 64 × 64 board, which answers in one lookup every distance needed by the generators, the route controller and the search. Measurement sets the split between search and table. The reply reserve is a high quantile of the measured reply cost, p99.99 plus a margin, with 6M points plus twice the costliest reply as a placeholder.

Recognising which top team's mimic this game's enemies resemble could give the enemy chance nodes better priors. The ladder's bots change, and unseen tournament maps may make matching harder, so it's tested late, against the final league, and kept only if it wins.

5. The offline teacher

The first network is kickstarted by imitating the top ladder teams' decisions in public replays, which the organisers allow. Each game is replayed in the judge from its match seed, so every dragon's observation, pearl countdowns included, is what it saw. Expert iteration then takes over, as AlphaStar learned from human play before its league (Vinyals et al., 2019).

From then on, the same search, run with full information and generous budgets on GPUs, produces improved policy targets (Anthony et al., 2017), using Gumbel draws to explore during self-play data collection. Every match uses zero draws, including the teacher check, where it has to beat the network choosing without search before its targets are used. A faithful simulator serves as the oracle there, never inside the bot. The teacher's value network takes privileged whole-board inputs and learns from the outcomes of search-played games, as in AlphaZero, and its values reach search leaves only after they pass a calibration check on held-out games.

The teacher's leaf value

The leaf value decides the teacher's strength. With the network's own value at its leaves, the teacher did no better than the plain network, and more simulations or depth didn't help. Its leaf log shows why: the network's value reads too optimistic at every depth, and its squared error against the outcome is worse than a constant's. Leaves played on for 20 rounds on the true state did far better:

The privileged value is fitted offline on recorded games so far, against the outcome, the n-step value 10 rounds on, and the team's shares of units, total length and longest dragon 10, 50 and 200 rounds on, and it passes the calibration check. Cycle 1's teacher values its leaves by 20-round rollouts ending on the privileged value, once a full-map check of hundreds of games confirms the small-map result. Their cost comes down by running a cheap rollout policy over every lane's dragons in one kernel, rolling out only from leaves whose value is uncertain, capturing the round loop as a CUDA graph and spreading games across GPUs. The teacher's simulations, depth and rollout length are set by strength gained per GPU-second and per dollar, while its root width is never cut.

Distilling the bot's value head

The bot's value head is distilled from the privileged value, as in asymmetric distillation (Warrington et al., 2021). At each of a dragon's decisions in training, on both teams' positions, it learns the fitted privileged value of the true state and the outcome. A training-only auxiliary head learns the team's shares 10, 50 and 200 rounds on, which shapes the shared body and is never exported. The value targets use TD(λ) instead of fixed horizons, with λ fitted on held-out calibration, and decisions that carry only value targets train no policy.

The privileged value's fit chooses its λ through a horizon. Samples are taken every K rounds, and a horizon of H rounds sets λ = 1 − K/H, so a target bootstraps H rounds ahead on average. Each horizon is one job, and the fit with the lowest calibration-set log loss among those that pass the calibration check is kept. Training stops when the held-out improvement over the best checkpoint so far, paired on the same decisions and with its standard error clustered by game, is no longer significant in a one-sided test at 95%.

The search reads the bot's value head only after it passes three checks: the held-out calibration check, a forked-continuation test, which copies recorded games at a random round and plays each copy to the end with the policy network so the head's values can be compared with the outcomes, and a match of the searching bot against the plain bot. The imitation-trained head fails the calibration check, because it learnt only from the top ten teams' sides, which won most of their games, so it reads too optimistic. The retrain on the new inputs replaces it.

Candidate, ranking and information gaps

A dragon sees only its 7×7 window, so targets chosen with information unavailable to it may be hard to imitate (Warrington et al., 2021). A student distilled from this teacher is therefore compared with one distilled at equal compute from a teacher whose determinisations come from the acting dragon's belief, and the teacher whose student wins is kept. Where the student still departs from the teacher, the departure is measured in value lost, the teacher's value of its move less its value of the student's. A fact observed or decided by some teammate earns a place in sonar when adding it to the student's inputs recovers that value, and facts are ranked by value recovered per bit of a dragon's 256 bits a turn. Departures beyond any teammate's knowledge are left to belief and search.

The teacher searches every legal primitive action as well as the generators' candidates, so each departure falls under exactly one of three causes. Without that, no departure can be called a candidate gap and no generator can be measured. A legal primitive action is a move, a legal split or a sprint, fatal ones included. A sprint is chosen first by its end cell and final heading, and its route is a second choice under that, admitted by progressive widening as the branch's visits grow, ordered by how different the resulting states are, with routes that leave the same state merged.

Each cycle reports value lost by cause, candidate gaps by situation and by generator, and for each generator how often its candidates are chosen, the value lost to other variants of its behaviour, its cost in bot points and self-play throughput, and its equal-cost ablation. A teacher is used only after it beats the plain network in a match whose size and pass bar are fixed before it runs.

Parallel turns

Most of the teacher's time goes to simulating the other dragons, whose moves come from the network. The rules play dragons one after another in ID order, each observing the effects of earlier moves, but dragon j's turn depends on an earlier mover i only when i's action changes one of j's inputs: a cell in j's view, j's inbox (i's rays, and later movers' rays now blocked or let through by i's segments), or a cell on j's own path. Each dragon casts at most four rays, and a ray stops at kelp or the first living segment, so a mover's messages reach at most four dragons.

Both schedules reproduce strict ID-order play exactly, provided each dragon has its own random stream. Every turn also reads its team's unit count, and any teammate's split or death changes it, so a purely conservative schedule waits on every earlier teammate. A hybrid that waits only on movers whose sprint or rays could reach a dragon does much better, but needs that reach computed for every pending mover:

Turns played a wave over 1,500 rounds of eight queen-era games

Schedule	Turns per wave
Conservative	at most 1.70
Hybrid	6.28
True dependencies	7.93

The implemented scheduler avoids computing reach. It chooses moves for the next eight dragons in a lane from the same starting state, then applies them in ID order, and a dragon whose turn block has changed by then ends the wave:

The scheduler's trade-off by movers considered a wave (loong-turn-dependencies)

Movers considered	Turns per wave	Discarded decisions per turn
4	3.34	0.17
8	4.60	0.63
16	4.92	1.82

It applies to every network-chosen turn, while the teacher still searches its own decisions in order. A persistent kernel for each block of games also plays step after step on the device without returning to the host. Both are checked by choice traces byte-identical to the ID-order teacher on recorded seeds.

Self-play also searches only a share of decisions in full and plays the rest by the network, which is playout cap randomization (Wu, 2019, for KataGo): more games for the same compute, while the searched decisions keep their full targets. Unlike the two methods above, it changes the training data, so its share is set by measured strength per GPU-hour, comparing networks distilled from equal compute at different shares.

6. League training

The network trains against the frozen pool, the foil and mimics of top ladder teams, in the style of AlphaStar's league (Vinyals et al., 2019), so the policy doesn't overfit to itself. The mimics come from the same imitation that kickstarts the first network, and no mimic is ever submitted as it stands. Games are played on generated maps matched to the official distribution, because tournament maps are unseen, and their network-chosen turns use part 5's parallel simulation. Registered builds such as the foil's run their WebAssembly on the CPU beside the CUDA engine, whose games match the official SDK byte for byte.

7. Decision review

The viewer shows for every decision the candidates, the network's values and the search's verdict, and the teacher review adds the played, network and teacher choices with each one's option, Q and value lost. Aggregate numbers say whether the bot wins, while individual decisions show why, which is how a wrong belief, a missing option or a misjudged value is found. The value-lost report points the review at the decisions that cost the most.

Checks

Each stage passes a match or check whose size and pass bar are fixed before it runs, so a lucky run can't be chosen after the fact. The ladder stays empty until a bot implementing the full expert design is at least as good as foil-0039 in a local match.

Strategic improvement from review

A learned bot can still make plainly wrong choices, such as not playing a move that kills the enemy queen. Each reviewed mistake is classified by where the right move was lost, because each place has a different fix. A move that was never offered is fixed in the hand-written generators, and such a logical fix, a change that stops the bot throwing a game in a specific situation, goes live once it builds and a few smoke games show no failed turns, without the statistical check for a learned change.

Proven outcomes in search

In the bot's search and the teacher's, a move proven by the rules to win or lose outright is marked proven instead of averaged, by MCTS-Solver (Winands, Björnsson and Saito, 2008). A proven win is always played and a proven loss never is, whatever the network says. The tree holds the searching dragon's own decisions, with other dragons' moves and the hidden state sampled between them, so a proof can't pass through a transition whose other outcomes were never seen. That leaves two sound kinds of proof: a root certainty, a move with an outcome fixed by the rules whatever is hidden, such as one that certainly kills the team's last dragon, and a proof over a fully determined segment, where nothing is hidden and no other dragon's move is sampled, such as an endgame with every relevant dragon in view. A queen kill doesn't end the game, but it decides the round-500 comparison while our queen lives, so it belongs to the value's targets and the reviewed positions instead of the solver.

Reviewed positions

The review library keeps each flagged position with its correct move and explanation, and every new bot and network version is checked against it, as a regression suite does for code. The same positions enter training with extra weight, labelled by the teacher, because a network trained on millions of ordinary positions barely notices getting a rare decisive one wrong, which is the idea behind hard-example mining (Shrivastava, Gupta and Girshick, 2016).

Budget

The judge's meter prices every instruction by its opcode, and every SIMD instruction costs 2 points whatever its lanes. Nothing else counts, not even a cache miss, so a branch-free kernel's cost is an exact function of its shape and a design's cost is known before it's trained. Optimising means using fewer, wider instructions. Measurements here were taken in the official SDK 1.2.7 compiler and sandbox, which build and run bots exactly as the online judge does, on help.map, the largest map at 64 × 64, with seed 7 fixing its pearls and the bots' random draws.

The compiler adds two points of address arithmetic to every load through a pointer. When the weights are a const array filled by C23's #embed and read at constant indices, it turns each load into a v128.const immediate, so a dot costs 5 points: the constant, the input from a local and the dot. That makes the ceiling 3.2 multiply-adds a point, before each output's reduction and store. The SDK's compiler and sandbox run the relaxed dot, but the online judge's acceptance of it is untested, because a test upload activates a submission.

Measurement	Value
Probe network, 2.8M random int8 weights, about 9.7M multiply-adds	4.49M points an evaluation, 2.5–2.65 multiply-adds a point per layer including requantisation
expert-0001's turn with 128 states a filter, averaged over a whole game of 30,554 turns	17.4M points
4 MB submission, with trained int8 weights deflating to 67%	about 5.7M trained weights

expert-0001's average turn on help.map with 128 states a filter

Component	Average points per turn
Network	5.1M
Filters' prediction	3.2M
Reply's write	3.0M
Listing candidates	2.1M
Crop's encoding	1.0M
Overview, distances, pearl chances and the rest	2.9M

That leaves the network and search about 88M points a turn before the reply reserve, enough for about 130 32-channel trunk layers. A search node costs only its encoding and one network evaluation, because its leaves start from the root's belief, predicted once a turn.

The network is close to both bounds. Everything else sits much further above its bound than its floor, so its remaining gaps need cheaper algorithms, such as incremental encodings and an overlay instead of a whole-game copy for each search sample.

Game knowledge

Replay review and opponent study taught these lessons, and the options, rewards and review should account for them:

• Under the earlier rules, the foil started most of its head-on collisions, many by sprint, so a threat model has to include the sprint behind a head.
• Under queen scoring, feeding consolidates into the queen, and recombination timing needs to be learned from its effect on queen survival and final length.
• A child split directly behind its parent can receive a map upload over sonar.
• Re-entering a farmed pocket that already holds teammates crowds them and can kill one, and two teammates can seal each other into a pocket two moves deep.
• A 1×1 dead end is rarely productive.
• Teammates' lengths can often be deduced exactly from observed splits and moves, and a dragon inside our own view can't be where we believe a teammate is.
• Hard role holds should give way to a decaying persistence, so desperate need can still win.
• Arrive at a spawning tile when its attempt is due, instead of chasing pearls already there.
• An enemy that wants to live dodges a straight strike, so aim at the inner corner of its likely escape.
• A route can collect pearls and reveal unseen ground at once, so collecting and exploring are values of one route.
• Anaconda ring: a long dragon encloses a large space as a ring around a farming area, then turns inside itself. It stays one sprint from safety, and an enemy caught inside costs a cheap split, but it can't send sonar from inside.
• Tail lurker: a small dragon waits at the enemy queen's tail to take the head of any dragon split off from it.
• Rearguard: a length-2 dragon follows our queen and sends sonar of its rear, showing its escape route and where a rival's split could come from.

Explored ideas

These methods were studied, and several built, while the bot's training was designed, most in the reinforcement-learning (RL) line that preceded expert iteration. The RL line trained a policy by trial and error with PPO, and its larger, more elaborate networks didn't turn into strength, so the current network started small and plain to get the expert-iteration loop built and measured first. Each idea is a candidate for an equal-cost comparison in a part's decision record.

Concept index

Every concept explained on this page, in order of first appearance.

• Partial observability and decentralisation (the game)
• Reinforcement learning in one page, expert iteration and CTDE (the design)
• Floors and bounds (principles)
• Encoding, planes and channels, Bayes filters, histogram, Bernoulli and renewal filters, encryption and check bits, KL divergence and KLD-sampling, calibration and informativeness (part 1)
• Policy and value heads, crops, stems and trunks, set encoders and attention, quantisation to int8, WebAssembly, SIMD and the relaxed dot, efficiently updatable networks (part 2)
• The options framework, skills, tactics and plays (part 3)
• Monte Carlo tree search and PUCT, determinisation and information-set MCTS, Gumbel root search and sequential halving, chance nodes, POMCP, public belief states: ReBeL and Student of Games, progressive widening, SPSA and CLOP (part 4)
• Imitation learning, distillation and cross-entropy, value functions, rollouts and privileged critics, TD(λ), the imitation gap, parallel discrete-event simulation, kernel launches, CUDA graphs and persistent kernels (part 5)
• League training (part 6)
• Wilson bounds (part 7)
• MCTS-Solver (strategic improvement)

Sources

Every method on this page, grouped by where it sits.

The overall method

• Anthony, Tian and Barber, Thinking Fast and Slow with Deep Learning and Tree Search (2017): expert iteration.
• Silver et al., Mastering the game of Go with deep neural networks and tree search (Nature, 2016): AlphaGo.
• Silver et al., Mastering the game of Go without human knowledge (Nature, 2017), and A general reinforcement learning algorithm that masters chess, shogi and Go through self-play (Science, 2018): network-guided search and its policy and value targets.
• Lowe et al., Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017), and Yu et al., The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (2022): centralised critics with decentralised actors.
• Kaelbling, Littman and Cassandra, Planning and acting in partially observable stochastic domains (Artificial Intelligence, 1998): POMDPs and belief states.
• Bernstein, Givan, Immerman and Zilberstein, The complexity of decentralized control of Markov decision processes (Mathematics of Operations Research, 2002): decentralised POMDPs and their hardness.
• Sutton and Barto, Reinforcement Learning: An Introduction, 2nd edition (MIT Press, 2018): reinforcement learning, policies, values and critics.

Belief and sonar (part 1)

• Thrun, Burgard and Fox, Probabilistic Robotics (2005): Bayes filters.
• Ristic, Vo, Vo and Farina, A Tutorial on Bernoulli Filters (IEEE Transactions on Signal Processing, 2013): a target's chance of existing.
• Feller, An Introduction to Probability Theory and Its Applications, volume 1, chapter 13: renewal processes.
• Rabiner, A Tutorial on Hidden Markov Models (Proceedings of the IEEE, 1989): the pearl's two-state model.
• Fox, Adapting the Sample Size in Particle Filters Through KLD-Sampling (IJRR, 2003): sizing a filter by a bound on its approximation error.
• Kullback and Leibler, On information and sufficiency (Annals of Mathematical Statistics, 1951): the divergence behind the filters' error bound.
• Guo, Pleiss, Sun and Weinberger, On calibration of modern neural networks (ICML, 2017), and Gneiting and Raftery, Strictly proper scoring rules, prediction, and estimation (JASA, 2007): calibration, ECE and scoring the belief.
• Beaulieu, Shors, Smith, Treatman-Clark, Weeks and Wingers, The SIMON and SPECK families of lightweight block ciphers (IACR ePrint 2013/404): sonar's cipher.
• Golomb, Run-length encodings (IEEE Transactions on Information Theory, 1966), and Teuhola, A compression method for clustered bit-vectors (Information Processing Letters, 1978): the sonar records' exponential-Golomb fields.

The network (part 2)

• Jacob et al., Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (CVPR, 2018): quantisation-aware training.
• Nasu, Efficiently Updatable Neural-Network-based Evaluation Functions for Computer Shogi (2018): NNUE, the incrementally updated first layer.
• Bengio, Léonard and Courville, Estimating or propagating gradients through stochastic neurons for conditional computation (2013): the straight-through estimator.
• Zaheer et al., Deep Sets (NeurIPS, 2017), and Vaswani et al., Attention is all you need (NeurIPS, 2017): reading the planned enemy tokens.

Options (part 3)

• Sutton, Precup and Singh, Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning (Artificial Intelligence, 1999): options.
• Browning et al., STP: Skills, tactics and plays for multi-robot control in adversarial environments (2005): cooperative plays.

Search (part 4)

• Coulom, Efficient selectivity and backup operators in Monte-Carlo tree search (Computers and Games, 2006), and Kocsis and Szepesvári, Bandit based Monte-Carlo planning (ECML, 2006): Monte Carlo tree search and UCT.
• Auer, Cesa-Bianchi and Fischer, Finite-time analysis of the multiarmed bandit problem (Machine Learning, 2002): UCB1.
• Rosin, Multi-armed bandits with episode context (Annals of Mathematics and Artificial Intelligence, 2011): the predictor-weighted bound behind PUCT.
• Cowling, Powley and Whitehouse, Information Set Monte Carlo Tree Search (IEEE TCIAIG, 2012): search under hidden information.
• Frank and Basin, Search in games with incomplete information: a case study using Bridge card play (Artificial Intelligence, 1998): strategy fusion.
• Long, Sturtevant, Buro and Furtak, Understanding the success of perfect information Monte Carlo sampling in game tree search (AAAI, 2010): determinisation.
• Danihelka, Guez, Schrittwieser and Silver, Policy improvement by planning with Gumbel (ICLR, 2022): policy improvement from few simulations.
• Karnin, Koren and Somekh, Almost optimal exploration in multi-armed bandits (ICML, 2013): sequential halving.
• Kool, van Hoof and Welling, Stochastic beams and where to find them: the Gumbel-top-k trick (ICML, 2019): sampling without replacement by Gumbel noise.
• Couëtoux et al., Continuous Upper Confidence Trees (LION, 2011): progressive widening's exponent form.
• Spall, Multivariate Stochastic Approximation Using a Simultaneous Perturbation Gradient Approximation (IEEE TAC, 1992), and Coulom, CLOP: Confident Local Optimization for Noisy Black-Box Parameter Tuning (2011): tuning constants jointly from noisy matches.
• Winands, Björnsson and Saito, Monte-Carlo tree search solver (Computers and Games, 2008): proven outcomes in search.
• Silver and Veness, Monte-Carlo Planning in Large POMDPs (NeurIPS, 2010): POMCP, a contender to information-set MCTS.
• Brown, Bakhtin, Lerer and Gong, Combining deep reinforcement learning and search for imperfect-information games (NeurIPS, 2020): ReBeL, search over public belief states, a contender.
• Schmid et al., Student of Games: a unified learning algorithm for both perfect and imperfect information games (Science Advances, 2023): growing-tree CFR, a contender.

Teacher and training (part 5)

• Schrittwieser et al., Mastering Atari, Go, chess and shogi by planning with a learned model (Nature, 2020): MuZero, the setting of Gumbel MuZero.
• Warrington et al., Robust Asymmetric Learning in POMDPs (ICML, 2021): the imitation gap of privileged teachers.
• Pinto, Andrychowicz, Welinder, Zaremba and Abbeel, Asymmetric actor critic for image-based robot learning (RSS, 2018): a critic with privileged state for an actor without it.
• Wilson, Probable inference, the law of succession, and statistical inference (JASA, 1927): the Wilson score interval.
• Sutton, Learning to predict by the methods of temporal differences (Machine Learning, 1988): TD(λ).
• Hinton, Vinyals and Dean, Distilling the knowledge in a neural network (2015): distillation.
• Kingma and Ba, Adam: a method for stochastic optimization (ICLR, 2015): Adam.
• Loshchilov and Hutter, Decoupled Weight Decay Regularization (ICLR, 2019): AdamW.
• Pascanu, Mikolov and Bengio, On the difficulty of training recurrent neural networks (ICML, 2013): clipping gradients by norm.
• Shrivastava, Gupta and Girshick, Training region-based object detectors with online hard example mining (CVPR, 2016): weighting the examples a model gets wrong.
• Chandy and Misra, Distributed simulation: a case study in design and verification of distributed programs (IEEE Transactions on Software Engineering, 1979): conservative parallel discrete-event simulation.
• Jefferson, Virtual time (ACM Transactions on Programming Languages and Systems, 1985): Time Warp, optimistic parallel discrete-event simulation.
• Wu, Accelerating Self-Play Learning in Go (2019): KataGo's playout cap randomization.

League and ratings (part 6)

• Vinyals et al., Grandmaster level in StarCraft II using multi-agent reinforcement learning (Nature, 2019): league training.
• Berner et al., Dota 2 with large scale deep reinforcement learning (2019): OpenAI Five.
• Schulman, Wolski, Dhariwal, Radford and Klimov, Proximal policy optimization algorithms (2017): PPO, a policy-gradient reinforcement-learning method.
• Elo, The Rating of Chessplayers, Past and Present (Arco, 1978): Elo ratings.

Precedents in similar games

• Lux AI Season 1 (Kaggle, 2021), won by Toad Brigade with deep reinforcement learning, per arXiv 2301.01609: a reference result for learned agents in multi-unit grid games.
• Kaggle Hungry Geese (2021), a four-player snake game on a wrapped board, won by DeNA's HandyRL team (DeNA; HandyRL), and the closest published precedent to Loong. The team trained a CNN, designed so convolutions respect the wrapped board, by distributed off-policy deep reinforcement learning in self-play (a participant's review), and at play time ran MCTS with a linear evaluation function, ensembled with a large network (a competitor's survey of Kaggle simulation competitions).

Explored ideas

• Jaderberg et al., Reinforcement learning with unsupervised auxiliary tasks (ICLR, 2017): UNREAL.
• Gloeckle et al., Better and faster large language models via multi-token prediction (ICML, 2024): multi-token prediction.
• DeepSeek-AI, DeepSeek-V3 technical report (2024): FP8 training, multi-token prediction and DeepSeekMoE.
• DeepSeek-AI, DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence (2026): Muon, FP4 quantisation-aware training of expert weights, and specialists distilled on-policy into one model.
• DeepSeek-AI, DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning (2025), and Shao et al., DeepSeekMath (2024): GRPO.
• Shazeer et al., Outrageously large neural networks: the sparsely-gated mixture-of-experts layer (ICLR, 2017); Puigcerver et al., From sparse to soft mixtures of experts (ICLR, 2024); Obando-Ceron et al., Mixtures of experts unlock parameter scaling for deep RL (ICML, 2024).
• Perez, Strub, de Vries, Dumoulin and Courville, FiLM: visual reasoning with a general conditioning layer (AAAI, 2018).
• Buro, From simple features to sophisticated evaluation functions (Computers and Games, 1999): Logistello's patterns; Szubert and Jaśkowski, Temporal difference learning of n-tuple networks for the game 2048 (IEEE CIG, 2014).
• Rusu et al., Policy distillation (ICLR, 2016); Teh et al., Distral: robust multitask reinforcement learning (NeurIPS, 2017); Agarwal et al., On-policy distillation of language models (ICLR, 2024).
• Micikevicius et al., FP8 formats for deep learning (2022).
• Zobrist, A new hashing method with application for game playing (1970): transposition keys.
• Schulman et al., High-dimensional continuous control using generalized advantage estimation (ICLR, 2016); Huang et al., The 37 implementation details of proximal policy optimization (2022); Ng, Harada and Russell, Policy invariance under reward transformations (ICML, 1999).
• Ross, Gordon and Bagnell, A reduction of imitation learning and structured prediction to no-regret online learning (AISTATS, 2011): DAgger; Rajeswaran et al., Learning complex dexterous manipulation with deep reinforcement learning and demonstrations (RSS, 2018): DAPG; Hester et al., Deep Q-learning from demonstrations (AAAI, 2018); Schmitt et al., Kickstarting deep reinforcement learning (2018); Kapturowski et al., Recurrent experience replay in distributed reinforcement learning (ICLR, 2019): R2D2.
• Jiang, Grefenstette and Rocktäschel, Prioritized level replay (ICML, 2021); Cobbe et al., Leveraging procedural generation to benchmark reinforcement learning (ICML, 2020); Jaderberg et al., Population based training of neural networks (2017), and Human-level performance in 3D multiplayer games with population-based reinforcement learning (Science, 2019).
• He, Zhang, Ren and Sun, Deep residual learning for image recognition (CVPR, 2016); Ba, Kiros and Hinton, Layer normalization (2016); Wu and He, Group normalization (ECCV, 2018); Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation (EMNLP, 2014): the GRU; Parisotto and Salakhutdinov, Neural Map (ICLR, 2018); Lee et al., SimBa (ICLR, 2025); Nauman et al., Bigger, regularized, optimistic (NeurIPS, 2024): BRO; Fedus, Zoph and Shazeer, Switch Transformers (JMLR, 2022).
• Bacon, Harb and Precup, The option-critic architecture (AAAI, 2017); Vezhnevets et al., FeUdal networks for hierarchical reinforcement learning (ICML, 2017); Lin et al., Don't use large mini-batches, use local SGD (ICLR, 2020); Douillard et al., DiLoCo: distributed low-communication training of language models (2023).
• Jordan et al., Muon: an optimizer for hidden layers in neural networks (2024), and Liu et al., Muon is scalable for LLM training (2025).