Handling large amounts of players

Handling large amounts of players.
image by jcomp - www.freepik.com

The handling of large amounts of concurrent players represents a central challenge in the implementation of an MMORPG project. At your hands are many different strategies. Read how to divide and conquer!

The present article has a similar topic to the article on scalable MMORPG server architecture. The latter is about distributed software systems. The present article is specific to the problem of large player numbers. Also, you might want to read about parallel programming basics first.

Table of Contents


Why are large amounts of players a problem?

In general, when x entities interact in a virtual space, the complexity of interactions can increase with x². Imagine 100 players on a market place, 50 casting rain-of-fire on the place, and the other 50 casting mass-heal.

CPU limit

The server that runs the game loop has a fixed time interval for this task. About 50 milliseconds are not uncommon. More players add more work to the loop.

So, how can you compensate? You can buy faster CPUs. However, this increase is linear, while the problem is exponential, so you run into limitations fast.

You cannot make the server loop arbitrarily longer, without threatening the illusion of continuous gameplay. Also, you cannot simply share the work of the loop because of the shared memory dilemma.

Network limit

Similar to the CPU limit, more players mean an exponential increase in network traffic. The solution here is to share the load between an arbitrary number of independent sending machines as described in the article on scalable MMORPG server architecture.


Strategies to handle large amounts of players

There are different strategies to deal with this: Brute Force, Peer-to-peer, Field of View, Phasing, Zoning, Dissociating, Message Optimization, Limiting Actions. You will need most of them.

Brute force

Brute force relies on rapid progress.
image by pikisuperstar – www.freepik.com

Technology constantly expands the limits of what is feasible. The brute force approach uses high-performance server hardware and relies on better and better Internet connections for clients.

Advantage

You don´t need to change the existing implementation. Technical progress can reduce costs and increase performance.

Disadvantage

Working at the limits of what is technically feasible is costly. The possible increases in performance are linear, while the requirements increase exponentially. In the end, you can achieve only limited effects with this approach.

Also, you may run into unexpected behavior when the hardware is very much better than that what the code was written for.

Peer-to-peer

A Peer to Peer non-authoritative architecture.
A Peer to Peer non-authoritative architecture.

With this approach, the server doesn’t send each player’s location to everyone else. Each player sends his position to the other. The load in the network is distributed to all clients. As a result, the data traffic no longer increases exponentially per server, but linearly with the number of clients per client.

Advantage

The decentralized approach enables a large number of concurrent players.

Disadvantage

The approach violates the three golden rules of multiplayer safety. Therefore, forget about using it with clients. However, when all peer nodes are server-side and under our control, this architecture is quite useful and commonly found (see scalable MMORPG server architecture).

Phasing

Phasing in MMORPGs.
Phasing distributes players evengly in world instances (= phases).

The server instantiates a new phase when player numbers in the old instances are too high. New players come to the new instance.

Also, games use phasing to apply persistent changes to the game world as a way of story-telling.

Advantage

You can precisely define the maximum amount of data traffic per instance. All traffic is divided by the number of instances.

Disadvantage

The approach damages the illusion of a persistent world, immersion, and the possibility of interacting with other players.

Zoning

Zones handle players according to their spatial distribution.

Zoning puts players on different servers depending on their spatial distribution. Implementation of this strategy is trivial when transition from zone to zone requires a loading screen.

The seamless zone transition of mobs requires elaborate mechanisms described here.

You should implement mechanisms to prevent a zone from being overloaded with players. The best way is by evenly distributing regions of interest.

Advantage

The approach can handle many evenly distributed players.

Disadvantage

Doesn’t work if there are many players in one place.

Limiting Actions

Limiting player actions to reduce traffic.
Limiting player actions to reduce traffic.

The position data are the characteristics of a mob that are constantly changing. Besides, mobs can have many more changeable properties: life points, mana, which action is currently being carried out, change of equipment, DoTs, HoTs, changes to attributes.

All of these properties need to be communicated to the players in the area. This can drive traffic up significantly. Many of these properties only change in combat or while using spells.

The strategy for limiting actions is therefore to avoid combat actions in areas with a high density of players. This can be designed in such a way that, for example, no attackable creatures can be found in cities. This will feel natural.

A more rigid variant could also deactivate all spells in cities (“anti-magic field”). In this scenario, you completely avoid fast-paced player interactions. Chat and trade are not fast-paced. Players buying from merchants or crafting stuff aren’t interactions between players.

Advantage

Data traffic is significantly reduced.

Disadvantage

Players might perceive it as limiting. High traffic stays a problem.

Dissociating

Parallel reading, processing, and writing of game state are possible when entities don´t affect each other.

Normally, players can affect each other. Therefore, processing the next game state on the server involves writing to shared common memory and needs to happen sequentially (read here).

However, when players don´t affect each other, you can change the server code so that also write operations are parallel.

Imagine a bottleneck zone that is highly populated by players. There are a number of resource nodes that are private for each player. That means when one player loots them, they stay untouched for the other players.

Dissociating is similar to Limiting actions, yet not the same. The latter doesn´t allow spells. The former allows spells, yet not in a way that other players are affected.

Advantage

Tremendous freedom of scalability because sequential write operations are a bottleneck of game state processing.

Disadvantage

The same as Limiting Actions.

Interest Management

Interest Management: Players (dark blue) can only see the creatures (gray) within their vision range (dotted circle).
Players (dark blue) can only see the creatures (gray) within their vision range (dotted circle).

Send only information about the players and creatures that exist within a certain range around the player. Sophisticated variants of the technology send information about other units in a graduated manner so that targeted and nearby units receive frequent updates and remote units only rarely.

Advantage

The most important strategy that significantly reduces traffic.

Disadvantage

Requires the implementation of a spatial grid. Doesn’t work if many players are standing in one place. Heavily overused but most games with the effect of mobs popping out of nowhere and the empty landscape effect.

Message Optimization

It is worth to reduce the message size.

About 95% of all messages sent by an MMORPG server are the position data of mobs. It is, therefore, worthwhile to reduce the size of this type of message as much as possible. Read more about message optimization here.

Advantage

Without changing any other major architecture feature you can reduce traffic by 2 to 10 fold.

Disadvantage

Message optimizations has limited gain.

Action prediction

image by upklyak – www.freepik.com

Action prediction is an extreme form of Message optimization that happens client-side. It is useful to handle mass battles with literally hundreds or even thousands of players.

In such situations, you can generously drop details about anything that is not in the closest range to the player. Action prediction affects these mobs.

It is enough to give each mob an id and position vector. Send updates only about these and only when they are significantly different than the old values to keep the message size minimal.

Even, when these entities cast spells all the time, you don´t need to send that information to 100 %. It is enough to send a small percentage.

The client can keep a history of spells that different entities cast. Apply a simple heuristic or a sophisticated AI to predict what other entities that are similar in type or situation may cast. Show that!

As these entities are not in close combat range and the player doesn´t know anything apart from their position and model, the player will not notice the cheat.


Seamless zone transition

Zone borders

The simplest incorporates fixed zone borders. In a use case with few NPCs and many players you might need flexible zone borders that adjust to player clumps.

Around each zone, there is an overlap. The overlap is larger or at least the same size as the player visibility and interaction range regarding other MOB.

Therefore, a player standing at the very border of a zone can watch into another zone and everything he sees or might interact with is within the zone border.

A zone owns the mobs that are within its borders. A mob has certain properties, depending on its location in the zone.

In the zone the mob exists once. The zone has the following tasks:

  • process status change of the mob

Mob in the zone periphery exists as original and as ghosts in the adjacent zone (consider corner cases with two adjacent zones and two ghosts). For periphery mobs the zone has the following tasks:

  • process status change of the original
  • send the original mob status to all ghosts to keep them identical to the original
  • receive status change requests from the adjacent zones and decide whether to apply them or not
  • hand mob ownership to the adjacent zone when a mob leaves
  • accept mob ownership from adjacent zones when a mob enters

In the overlap mobs exist as ghosts (the original is in the adjacent zone). For overlap mobs the zone has the folowing tasks:

  • store where the original mob is
  • update the ghosts with status updates from the adjacent zone with the original
  • forward status change requests to the original mob (e.g. players shoot at mobs in the overlap)

Posession and ghosts

The zone periphery marks the area of a zone that is covered by the zone border of an adjacent zone. It is the task of each zone to send updates of their creatures that are in the zone periphery to the adjacent zones.

MMORPG seamless zone transition: Player P in the middle of the blue zone can see creature C2.
MMORPG seamless zone transition: Player P in the middle of the blue zone can see creature C2.

The player P and its interaction range (dotted line) are shown. C2 is a creature within the interaction range. Therefore it is visible. All other creatures are invisible to P.

C3 is in the periphery of zone B. Therefore, zone B processes all changes to C3. At the same time, C3 is a ghost in the overlap of zone A. Every change to C3 is also updated to the overlap of zone A to be correctly displayed

Player casting across zone borders

Every change request of the C3 ghost is forwarded to C3. So when P casts a spell at C3, the C3 ghosts forwards this spell to zone B. Zone B applies the spell to C3. The C3 ghost is updated according to the changes that the spell did to C3.

MMORPG seamless zone transition: C3 exists as a ghost in the overlap of the blue zone and as original in the periphery of the grey zone.
MMORPG seamless zone transition: C3 exists as a ghost in the overlap of the blue zone and as original in the periphery of the grey zone.

In a worst-case scenario, C3 moves across the zone border into zone A while the spell is cast. So the spell is forwarded to zone A. However, when the spell arrives in zone A, C3 is only a ghost of zone B. So the spell is again forwarded to the owner, which is zone A now.

Player moving across zone borders

MMORPG seamless zone transition: When Player P crosses the border he becomes the property of the grey zone.
MMORPG seamless zone transition: When Player P crosses the border he becomes the property of the grey zone.

So when the player crosses the border to zone B, ownership goes to zone B. The mechanism is the same as for creatures. You may notice that each cross-border interaction requires data transfer between zone nodes. Therefore, create zones in a way to minimize player and mob activity in the zone peripheries.


Checklist

Handling large amounts of players requires to combine a number of strategies:

Also, seamless zone transition is the standard for very large game worlds. The implementation is not trivial.


Server Sid
Server Sid

Regarding player numbers, not everything that is technically possible is sensible. Rather, design considerations should decide.

(Last Updated on April 27, 2021)

1 comment

  1. I have read this article and the server architecture article three times and today I went through and took notes. I just wanted to thank you for the detailed explanations!

Leave a comment

Your email address will not be published.