The phrase “train against the real game” sounds like the gold standard — no simplified proxy environment, no abstraction leak, just the actual simulation core driving the live world. In practice it introduces hard problems that purpose-built training environments dodge.
The maintainer of world-of-claudecraft, an open-source browser MMO, walks through the friction points:
- Determinism is mandatory and expensive. Anything timing-dependent or tick-alignment-sensitive must be pushed out of the core or made explicit — otherwise rollouts can’t be reproduced. The work is worthwhile for netcode anyway, but it’s not free.
- Observation space is the unresolved problem. An MMO state is enormous and mostly irrelevant at any given moment. Feed the agent a structured summary and you lose the fidelity you wanted from the real game. Feed it raw state and you drown in dimensionality.
A commenter suggests two pragmatic cuts: layer abstractions over state (coarser representations, summary data — accept that the model will be “good enough, not optimal”), and filter the observation to only relevant channels (islanding — prune entities that can’t affect the current learning objective).
The post doesn’t offer a satisfying answer to the core tradeoff. That’s what makes it worth reading — it’s an honest signal from someone who actually built the thing, not a theoretical pitch.