The handling of large amounts of concurrent players represents a central challenge in the implementation of an MMORPG project. At your hands are many different strategies. Read how to divide and conquer!
The present article has a similar topic to the article on scalable MMORPG server architecture. The latter is about distributed software systems. The present article is specific to the problem of large player numbers. Also, you might want to read about parallel programming basics first.
Table of Contents
- Why are large amounts of players a problem?
- Strategies to handle large amounts of players
- Seamless zone transition
Why are large amounts of players a problem?
In general, when x entities interact in a virtual space, the complexity of interactions can increase with x². Imagine 100 players on a market place, 50 casting rain-of-fire on the place, and the other 50 casting mass-heal.
The server that runs the game loop has a fixed time interval for this task. About 50 milliseconds are not uncommon. More players add more work to the loop.
So, how can you compensate? You can buy faster CPUs. However, this increase is linear, while the problem is exponential, so you run into limitations fast.
You cannot make the server loop arbitrarily longer, without threatening the illusion of continuous gameplay. Also, you cannot simply share the work of the loop because of the shared memory dilemma.
Similar to the CPU limit, more players mean an exponential increase in network traffic. The solution here is to share the load between an arbitrary number of independent sending machines as described in the article on scalable MMORPG server architecture.
Strategies to handle large amounts of players
There are different strategies to deal with this: Brute Force, Peer-to-peer, Field of View, Phasing, Zoning, Dissociating, Message Optimization, Limiting Actions. You will need most of them.
Technology constantly expands the limits of what is feasible. The brute force approach uses high-performance server hardware and relies on better and better Internet connections for clients.
You don´t need to change the existing implementation. Technical progress can reduce costs and increase performance.
Working at the limits of what is technically feasible is costly. The possible increases in performance are linear, while the requirements increase exponentially. In the end, you can achieve only limited effects with this approach.
Also, you may run into unexpected behavior when the hardware is very much better than that what the code was written for.
With this approach, the server doesn’t send each player’s location to everyone else. Each player sends his position to the other. The load in the network is distributed to all clients. As a result, the data traffic no longer increases exponentially per server, but linearly with the number of clients per client.
The decentralized approach enables a large number of concurrent players.
The approach violates the three golden rules of multiplayer safety. Therefore, forget about using it with clients. However, when all peer nodes are server-side and under our control, this architecture is quite useful and commonly found (see scalable MMORPG server architecture).
The server instantiates a new phase when player numbers in the old instances are too high. New players come to the new instance.
Also, games use phasing to apply persistent changes to the game world as a way of story-telling.
You can precisely define the maximum amount of data traffic per instance. All traffic is divided by the number of instances.
The approach damages the illusion of a persistent world, immersion, and the possibility of interacting with other players.
Zoning puts players on different servers depending on their spatial distribution. Implementation of this strategy is trivial when transition from zone to zone requires a loading screen.
The seamless zone transition of mobs requires elaborate mechanisms described here.
You should implement mechanisms to prevent a zone from being overloaded with players. The best way is by evenly distributing regions of interest.
The approach can handle many evenly distributed players.
Doesn’t work if there are many players in one place.
The position data are the characteristics of a mob that are constantly changing. Besides, mobs can have many more changeable properties: life points, mana, which action is currently being carried out, change of equipment, DoTs, HoTs, changes to attributes.
All of these properties need to be communicated to the players in the area. This can drive traffic up significantly. Many of these properties only change in combat or while using spells.
The strategy for limiting actions is therefore to avoid combat actions in areas with a high density of players. This can be designed in such a way that, for example, no attackable creatures can be found in cities. This will feel natural.
A more rigid variant could also deactivate all spells in cities (“anti-magic field”). In this scenario, you completely avoid fast-paced player interactions. Chat and trade are not fast-paced. Players buying from merchants or crafting stuff aren’t interactions between players.
Data traffic is significantly reduced.
Players might perceive it as limiting. High traffic stays a problem.
Normally, players can affect each other. Therefore, processing the next game state on the server involves writing to shared common memory and needs to happen sequentially (read here).
However, when players don´t affect each other, you can change the server code so that also write operations are parallel.
Imagine a bottleneck zone that is highly populated by players. There are a number of resource nodes that are private for each player. That means when one player loots them, they stay untouched for the other players.
Dissociating is similar to Limiting actions, yet not the same. The latter doesn´t allow spells. The former allows spells, yet not in a way that other players are affected.
Tremendous freedom of scalability because sequential write operations are a bottleneck of game state processing.
The same as Limiting Actions.
Send only information about the players and creatures that exist within a certain range around the player. Sophisticated variants of the technology send information about other units in a graduated manner so that targeted and nearby units receive frequent updates and remote units only rarely.
The most important strategy that significantly reduces traffic.
Requires the implementation of a spatial grid. Doesn’t work if many players are standing in one place. Heavily overused but most games with the effect of mobs popping out of nowhere and the empty landscape effect.
About 95% of all messages sent by an MMORPG server are the position data of mobs. It is, therefore, worthwhile to reduce the size of this type of message as much as possible. Read more about message optimization here.
Without changing any other major architecture feature you can reduce traffic by 2 to 10 fold.
Message optimizations has limited gain.
Action prediction is an extreme form of Message optimization that happens client-side. It is useful to handle mass battles with literally hundreds or even thousands of players.
In such situations, you can generously drop details about anything that is not in the closest range to the player. Action prediction affects these mobs.
It is enough to give each mob an id and position vector. Send updates only about these and only when they are significantly different than the old values to keep the message size minimal.
Even, when these entities cast spells all the time, you don´t need to send that information to 100 %. It is enough to send a small percentage.
The client can keep a history of spells that different entities cast. Apply a simple heuristic or a sophisticated AI to predict what other entities that are similar in type or situation may cast. Show that!
As these entities are not in close combat range and the player doesn´t know anything apart from their position and model, the player will not notice the cheat.
Seamless zone transition
The simplest incorporates fixed zone borders. In a use case with few NPCs and many players you might need flexible zone borders that adjust to player clumps.
Around each zone, there is an overlap. The overlap is larger or at least the same size as the player visibility and interaction range regarding other MOB.
Therefore, a player standing at the very border of a zone can watch into another zone and everything he sees or might interact with is within the zone border.
A zone owns the mobs that are within its borders. A mob has certain properties, depending on its location in the zone.
In the zone the mob exists once. The zone has the following tasks:
- process status change of the mob
Mob in the zone periphery exists as original and as ghosts in the adjacent zone (consider corner cases with two adjacent zones and two ghosts). For periphery mobs the zone has the following tasks:
- process status change of the original
- send the original mob status to all ghosts to keep them identical to the original
- receive status change requests from the adjacent zones and decide whether to apply them or not
- hand mob ownership to the adjacent zone when a mob leaves
- accept mob ownership from adjacent zones when a mob enters
In the overlap mobs exist as ghosts (the original is in the adjacent zone). For overlap mobs the zone has the folowing tasks:
- store where the original mob is
- update the ghosts with status updates from the adjacent zone with the original
- forward status change requests to the original mob (e.g. players shoot at mobs in the overlap)
Posession and ghosts
The zone periphery marks the area of a zone that is covered by the zone border of an adjacent zone. It is the task of each zone to send updates of their creatures that are in the zone periphery to the adjacent zones.
The player P and its interaction range (dotted line) are shown. C2 is a creature within the interaction range. Therefore it is visible. All other creatures are invisible to P.
C3 is in the periphery of zone B. Therefore, zone B processes all changes to C3. At the same time, C3 is a ghost in the overlap of zone A. Every change to C3 is also updated to the overlap of zone A to be correctly displayed
Player casting across zone borders
Every change request of the C3 ghost is forwarded to C3. So when P casts a spell at C3, the C3 ghosts forwards this spell to zone B. Zone B applies the spell to C3. The C3 ghost is updated according to the changes that the spell did to C3.
In a worst-case scenario, C3 moves across the zone border into zone A while the spell is cast. So the spell is forwarded to zone A. However, when the spell arrives in zone A, C3 is only a ghost of zone B. So the spell is again forwarded to the owner, which is zone A now.
Player moving across zone borders
So when the player crosses the border to zone B, ownership goes to zone B. The mechanism is the same as for creatures. You may notice that each cross-border interaction requires data transfer between zone nodes. Therefore, create zones in a way to minimize player and mob activity in the zone peripheries.
Handling large amounts of players requires to combine a number of strategies:
- Brute Force and Peer-to-Peer server-side are basic technical strategies to cope with large amounts of players
- Phasing and Zoning split the player load
- Limiting Actions and Dissociation reduce the traffic independent of players’ distribution at the cost of reduced freedom of action
- Interest Management reduces the traffic dependent on the players’ distribution
- Message Optimization reduces traffic at the cost of accuracy
- Action prediction is a client-side strategy to cope with incomplete data
Also, seamless zone transition is the standard for very large game worlds. The implementation is not trivial.
Regarding player numbers, not everything that is technically possible is sensible. Rather, design considerations should decide.
(Last Updated on April 27, 2021)