AI for Warzone 2100 Masaryk University - Faculty of Informatics - IS MU
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Masaryk University
Faculty of Informatics
AI for Warzone 2100
Bachelor’s Thesis
Filip Ďuračka
Brno, Spring 2020Masaryk University
Faculty of Informatics
AI for Warzone 2100
Bachelor’s Thesis
Filip Ďuračka
Brno, Spring 2020This is where a copy of the official signed thesis assignment and a copy of the Statement of an Author is located in the printed version of the document.
Declaration
Hereby I declare that this paper is my original authorial work, which
I have worked out on my own. All sources, references, and literature
used or excerpted during elaboration of this work are properly cited
and listed in complete reference to the due source.
Filip Ďuračka
Advisor: RNDr. Mgr. Jaroslav Bayer
iAcknowledgements
I‘d like to thank my advisor Jaroslav Bayer for all the support, and
Warzone 2100 community for maintaining this amazing game for all
those years.
iiiAbstract The goal of this work was to design and implement an AI for real- time strategy game Warzone 2100. As part of the design, two new features unique to this AI were implemented. First is creation of a graph representation of any given game map, obtained by analysing terrain and application of image processing algorithms. Second is a neural network, trained on data from 727 matches against other AIs, that guides the AI‘s battle plan. iv
Keywords
Artificial Intelligence, Game AI, Image Processing, Artificial Neural
Networks, Warzone 2100, JavaScript
vContents
Introduction 1
1 Warzone 2100 3
1.1 Game mechanics . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Scripting language used for AIs . . . . . . . . . . . . . . . 6
1.3 Existing AIs . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Nexus AI . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.2 NullBot . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 Bonecrusher . . . . . . . . . . . . . . . . . . . . . 7
1.3.4 Cobra . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 TenguAI 11
2.1 Map analysis . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Terrain analysis . . . . . . . . . . . . . . . . . . . 12
2.1.2 Image thinning . . . . . . . . . . . . . . . . . . . 13
2.1.3 Graph creation . . . . . . . . . . . . . . . . . . . 16
2.1.4 Graph representation and manipulation . . . . . 18
2.2 Cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.1 Macromanagement cores . . . . . . . . . . . . . 21
2.2.2 Micromanagement cores . . . . . . . . . . . . . . 24
2.3 Neural network for playstyle selection . . . . . . . . . . . . 27
3 Conclusion 31
A Maps used for neural network training 33
B Statistics of matches before and after the use of neural net-
work 37
Bibliography 39
viiList of Tables
2.1 Visualisation of tile neighbourhood [8] 14
2.2 Match statistics during learning process, per
opponent. 29
2.3 Match statistics with learned neural network, per
opponent. 29
B.1 Match statistics during learning process, per map and
game setting. 37
B.2 Match statistics with learned neural network, per map
and game setting. 38
ixList of Figures
2.1 ’Picturisation’ of a map. Minimap (left) compared binary
picture (right). Red denotes impassable tiles. 14
2.2 Comparison of binary image (left) to its thinned
counterpart (right). 15
2.3 Thinned shape with nodes. Starting positions (green),
resources (blue), crossroads (brown). 16
2.4 Final graph of the map. Starting positions (green),
resources (blue), chokepoints (red), mundane
nodes(black). 17
A.1 Small map with tight corridors and perfect early game
chokepoint in the middle. 33
A.2 Water-separated continents with varying amounts of
players. Ideal for testing analyser and cross-water attacks
of the AI. 33
A.3 Sizeable, but tight map with hill-separated pathways.
Good for concentrated defenses and macro
movement. 34
A.4 Small but spacious map. Good for quick matches that do
not require fancy movements. 34
A.5 Personal favourite. Tight, highly asymmetric starting
positions. 35
A.6 Similar in concept to Monocot, but with more resources
and corridors. 35
xiIntroduction
The goal of this thesis is to create a new AI for Warzone 2100, that
would be both more challenging and fun to play against, and deeper
in its decision-making process than the alternatives. This is attempted
by two systems no other AI in this game has. Firstly, giving the AI
proper knowledge of the playing field by analysing the game map and
creating a graph which can then be traversed in an arbitrary fashion.
Secondly, equipping it with a neural network trained to select the best
playstyle for the current state of the game.
The first chapter introduces the game itself, explains the mechan-
ics and presents AIs currently in use for multiplayer games, their
strengths and weaknesses. The second chapter dissects design and
implementation of my AI, what it does better, faster, what it does extra
and how it affects an average game session. The end contains analysis
of the results of pitting the AIs against each other and discussion about
possible causes.
11 Warzone 2100
This chapter explains what the game is [1], what sets it apart from
most other games in its genre and design of AIs most widely used for
play.
1.1 Game mechanics
Warzone 2100 is a real-time strategy game set in a post-apocalyptic fu-
ture. As such, the goal of the game is to destroy all hostile assets present
on the map. To this end, players have to build a base of operations,
secure income of resources and produce an army, while defending
their own base against enemy attacks [2, 3].
The maps are made of square tiles organised into rectangular grid.
Each tile can be occupied by a single player-built structure or environ-
mental feature (e.g. trees, houses, miscellaneous ruins), with some
structures requiring several tiles. This grid has no effect on units, which
can roam freely across tiles. Maps can be up to 250 * 250 tiles in size.
Building a base is done via special, defenseless builder unit avail-
able since the start of the game. Typical player base would consist
of headquarters structure and some amount of factories, research
facilities, generators and static defenses(turrets, bunkers and walls)
positioned around the perimeter.
Resource collection in the game is simplified compared to others in
the genre. Instead of sending out defenseless and potentially expensive
harvester unit, Warzone 2100 makes the player build a free(and de-
fenseless) resource extractor on designated places on the map. These
are, upon construction, automatically assigned to a power genera-
tor, which generates the game‘s only resource – power – at speed
dependent on the amount of extractors assigned. Maximum number
of these a single generator can serve is four, so players must take care
in reserving enough space and power for multiple generators.
Unlike most games in this genre, Warzone 2100 provides the player
with an opportunity to design their own units, called droids. After
building the headquarters, and only while this building is present on
the map, the player gets access to a designer tool. Within this tool,
31. Warzone 2100
they can create a droid template by selecting components of three
categories:
∙ body, affecting base health, horsepower, thermal and kinetic
armour and turret capacity of the droid;
∙ propulsion, affecting speed, general mobility of the droid, type
of traversable terrain and resistance(or vulnerability) to certain
weapon types;
∙ turret, which is either a weapon, or a utility, i.e. constructor,
radar, repair.
In this tool, player can design aircraft by selecting a VTOL (Verti-
cal Take-off and Landing) propulsion. This changes a lot of proper-
ties of the droid. Firstly, and unsurprisingly, it gains the ability to fly
over any terrain and becomes untargetable by many ground-based
weapons. Secondly, it gains access to special VTOL only variants of
droid weapons (and some new weapons of their own) that have lim-
ited ammunition supply. Lastly, it needs a special factory to be pro-
duced in, and rearming pads to repair and replenish the ever needed
ammunition.
Role of ground infantry is held by cyborg units. These are, unlike
droids or VTOLs, not customisable, instead coming as template ready
for production when necessary technology has been researched. Basic
cyborg templates are unlocked together with certain droid weapons.
Heavier, more durable and damaging variations are available to be re-
searched separately after researching several high-tech droid weapons.
Cyborgs follow their own armor upgrade research path separate from
the droids and require special factory to be produced in. Related to cy-
borgs are flying unarmed transports capable of ferrying small amount
of cyborgs across otherwise impassable terrain, with heavy cyborgs
requiring their larger, more durable variant.
Like many contemporary RTS games, Warzone 2100 features re-
search opportunities to enhance capabilities of player‘s units. However,
unlike most of these games, research here provides much more than
upgrading stats. Within the very expansive technology tree, players
can unlock new buildings, base defenses and droid components in
addition to mentioned damage/armour/speed upgrades for all assets.
41. Warzone 2100
It is crucial for player‘s success to properly prioritise which research
path to follow, when to keep following it and when to switch to another
one [4].
Weapons are possibly the most important research items in the
whole tech tree. They are separated into several categories. These de-
termine what upgrades apply to the weapon in question, and modify
damage to certain propulsions and structure types [5]. Weapons them-
selves deal either kinetic or thermal damage that gets mitigated by
kinetic or thermal armour, respectively. Accuracy of the weapon also
plays a role, with bullets being able to completely miss their target
and hit something nearby, or detonate on terrain. Some weapons get
mild bonus to accuracy when firing from within half of their maxi-
mum range, which combined with possibility of hitting something
else makes proper unit placement in battle not negligible.
There are multiple parameters that can be set before the start of
each game. Aside from choosing a map for the game to take place
on, the host can also alter starting power for all players(low, medium,
high), starting bases(no base, basic structures only, defended base),
structure limits(except defensive structures) and starting technology
level(no tech, basic components available, advanced components avail-
able, everything researched).
The game contains basic diplomacy system. Its type can be set in
the lobby and can be one of:
∙ free for all, where every player stands for themselves;
∙ dynamic alliances, where players can form and break alliances
at will during the course of the game;
∙ locked teams, where the alliances are decided during game
setup;
∙ locked teams with shared research, which is self-explanatory.
When in alliance, players can choose to share map vision and send
power or droids to an allied player.
Last option during game setup is the presence of scavengers. These
are AI-controlled, non-playable faction that uses unique units and
structures. They are usually found guarding power sources around
51. Warzone 2100
the map, and may have a base with unit production capabilities in
a central location on a map. Their destruction is not necessary for
victory, but they do provide a sort of anti-rush protection and generally
slow the early game flow in games when enabled.
Last thing, there are two types of pick-ups available during the
course of a game. First is an oil drum, which provides immediate bonus
of 1000 energy upon pick up. Second one is an artifact, which some-
times spawns from a destroyed factory. When picked up, it unlocks
a component for research for the player, regardless of its dependen-
cies. This component is always such that the owner of the factory has
researched it, but the picking player has not. In case there is no such
component, artifact simply does nothing.
1.2 Scripting language used for AIs
All AI scripting is done in so called "Warzone 2100 JavaScript Scripting
API". This is, at least as far as we can tell, adhering to ECMAScript 5.1
specification. If the reader would find themselves scripting in this API,
caution should be taken to not attempt to use classes, "=>" operator
and some quality of life functions and consult ECMA 5.1 specification1
and API documentation2 frequently.
1.3 Existing AIs
This section introduces popular AIs available for play.
1.3.1 Nexus AI
Original AI made by Pumpkin Studios. Very simple in nature and
slow to react. Testing matches against Nexus almost always resulted
in its defeat, even when set on highest difficulty. Thus, Nexus will not
be used as an opponent.
1. http://web.archive.org/web/20111103184035/http://www.
ecma-international.org/publications/files/ECMA-ST/ECMA-262%
20edition%205.1%2C%20June%202011.pdf
2. https://github.com/Warzone2100/warzone2100/blob/master/doc/
Scripting.md
61. Warzone 2100
1.3.2 NullBot
NullBot3 is an AI featuring dynamic unit design and research based
on its opponent‘s choices.
There are 4 major personalities available for this bot: standard,
hover(optimised for watery maps), turtle(builds only defensive struc-
tures, no units but builders) and scavenger(takes control of scavenger
NPC faction). Each of these has subpersonalities defined by what
weapon types they focus on.
Adaptive mechanism gathers enemy unit statistics - which compo-
nents were used and guess their purpose. Based on this information, it
attempts to build an army equipped to maximum effectiveness against
that opponent.
During design process, weapons are picked by their expected role
and matched with body by its weight class.
Nullbot is also able to cooperate with human player if they happen
to be together in an alliance. By simple chat commands, player is able
to request power or units from the bot.
1.3.3 Bonecrusher
Non-cheating AI that favours aggressive expansion. Bonecrusher4 is
usually regarded as the most able of the available AIs [6].
Bonecrusher‘s droid design is rather simple – if available, it always
builds heavier bodies. Propulsion is always weighted to the heaviest
one it can afford. It chooses weapons based on how much research
went into their type, and from the most researched type it chooses one
of x best options at random. It completely ignores anti-air turrets.
AI‘s research follows hard-coded paths, each providing a specific
benefit, be it upgrades to economy, armor or components. Weapon
type the AI uses for a game, and thus its main focus, is contained in
pre-defined profiles. These profiles are assigned at the beginning of
the game either at random, or to supplement other AIs in the team.
After this assignment, based on AI‘s difficulty, the profile‘s techs are
researched in one of three ways.
3. https://github.com/haoNoQ/nullbot
4. https://github.com/EuPhobos/bonecrusher
71. Warzone 2100
∙ In strict order, as defined in the profile, being supplemented by
upgrade paths;
∙ As ’smudged’, meaning each tech in a profile gets its own path,
consisting of a single research;
∙ Random order, where the profile gets ’smudged’ and its tech
order shuffled.
Smudging can cause certain technologies to be researched earlier than
in strict mode, potentially altering AI‘s design choices.
If the bot doesn’t consider itself rich, its VTOLs target enemy re-
source extractors, otherwise they bomb production facilities, or scout
last known enemy positions. Repair units and jammers either chicken
out towards nearest defensive structure when the army is dying or try
to get close to hurt units.
Partisans target:
1. enemy walls;
2. immediate threat to self;
3. immediate threat to ally if Bonecrusher considers itself rich;
If they are understrength, they run to repair units if available, else
they scout unknown resources. In case partisans consider themselves
in full strength, they harass resource locations, scout the map, target
factories and builders, in this order. Bonecrusher takes care to get
heavily wounded units repaired whenever possible.
Cyborgs harass enemy resources when understrength, else they
attack defenses and factories, but only if its own base is not under
threat. If they cannot do anything better, they clear map features (e.g.
trees, ruined miscellaneous buildings). If nothing of the above can be
done, they join regular army.
Regular droids first assemble at given point, avoiding combat when
overwhelmed. They get called when under attack.
The bot is characteristic in its aggressive early game resource ac-
quisition and liberal use of defensive structures. It attempts to build
defenses near every resource location, regardless of its current owner.
Exact amount of these defenses scales with difficulty, but it always
81. Warzone 2100
results in well fortified resource extractor, regardless if bonecrusher
got to the resource first or the defenses destroyed extractor already
present.
Much like Nullbot, Bonecrusher is capable of interaction with
player via chat messages. Aside from simple greetings and smug
trash-talk, it is capable of responding to requests for power, units, or
to pursue a certain profile. Of course, player needs to be allied with
the bot to use this feature.
1.3.4 Cobra
Cobra5 is optimised for shared research games.
It has personalities based on the combination of weapons it uses,
some of which are considered unusable if the AI plays alone.
Its unit design does not differentiate unit‘s purpose. All units are
designed as either, attackers, VTOLs or system(builder, repair or sen-
sor). It prefers to use heavy bodies when possible, tracks for propulsion
and weapon based on its personality.
From tactics standpoint, Cobra divides droids into ordinary attack-
ers and artillery. Attackers, cyborgs and VTOLs attack nearest assets
of an enemy perceived to be the biggest threat. Artillery acts similarly,
but works closely with sensors to take full advantage of its range.
5. https://github.com/KJeff01/Cobra
92 TenguAI
This chapter analyses our AI, it‘s core components and performance
in comparison to competition.
Design philosophy for this AI is comprised of several points.
Firstly, no-cheating policy (also know as basic decency). To main-
tain fair-play and truly test the capability of this AI, care is taken
to avoid usual crutches used in AI design, mostly having access to
information it‘s human counterpart would not have under identical
circumstances.
Secondly, independence from balance changes. In the spirit of low
maintenance, all components and their relative comparisons are not
to be assumed immutable. Any change of base values or addition/re-
moval of components should not negatively impact the usability of
the design process.
Thirdly, to differentiate this AI from competitors and give it the
technological edge, graph representation of the battlefield is to be
created and utilised. No other Warzone 2100 AI has succeeded in (and
perhaps even attempted) this task. As this is the most distinctive and
important feature of this AI, it is discussed as very first.
Fourthly, to stay more firmly within the field of AI, a neural network
is trained to choose (preferably) optimal playstyle for any given game
state the AI can find itself in.
In light of these points, a completely new AI was built from scratch
to accomodate the analysis and neural network.
2.1 Map analysis
If player is to make sound tactical and strategic decisions, knowl-
edge of the playing field is paramount. Starting positions, locations
of resources, chokepoints, environmental hazards, all of these are
important factors to include in decision making process.
In modern RTS games, all competitive AI designs have access to
a graph representation of the map.[7] This graph is usually created
by human map designer simultaneously with the map itself. War-
zone 2100, being as old game as it is, has no such feature, and by
having active map-making community, creating a graph for each map
112. TenguAI
individually is not viable. Instead, we opted for automatic map analy-
sis.
The analysis takes place at the end of level loading sequence, in
a state when environment is fully loaded but player objects have not yet
been instantiated. There would be an option to perform this analysis
at proper level start, but seeing this would freeze the game for the
duration of the analysis process (which takes around 3-4 seconds on
average), it was decided against.
The analysis process can be divided into several stages: terrain
analysis, shape thinning and graph creation.
2.1.1 Terrain analysis
During pre-processing, node objects are created for points of interest
(further as POIs): player starting positions, resource locations and
gateways. These gateways are places defined by the map creator, meant
to help AIs identify easily defendable areas. Such perfect knowledge of
the battlefield is not to be considered cheating, as players themselves
have access to map preview before every match. There, these points
(except gateways) are highlighted.
Next, a general map topology is found by examining what propul-
sion can reach which POIs. Wheels are used as a base, land-only
propulsion, hover as water-capable option. There are several purposes
to this information. Firstly, analysis itself uses it to determine what
it should consider traversable. Secondly, it limits the unit designer,
as to not use water-incapable propulsion on a water-dominated map.
Thirdly, it guides micro core‘s choice of unit compositions. The AI is
capable of recognising these topologies:
∙ If all POIs can be reached by wheels, the map is declared ’land’ -
designer is free to use all propulsions and any water present is
considered not traversable;
∙ If all players can be reached by wheels but some resources only by
hovercraft, ’semi-land’ - unit designer prefers to equip construc-
tion units with water-capable propulsions, water is traversable;
122. TenguAI
∙ If some players can only be reached by hovercraft, ’water’ - special
micromanagement subroutines are activated and water is, again,
considered a traversable tile;
∙ Otherwise, topology is declared ’vtol’ - only flying units will be
used and all tiles are considered passable.
2.1.2 Image thinning
To create representation of the map, knowledge of POIs alone does
not suffice. We also need information about edges between them. This
could be done by running simultaneous BFS searches from each POI,
and connecting nodes who meet each other. This approach, however,
omits information about crossroads, passageways, even whole alter-
nate routes. Instead, we can interpret the map as an image. Images
have shapes. And shapes can be minimised down to simple lines,
along which we can place edges of the graph.
Actual goal of this stage is to obtain medial axis transform, or its
close approximation, of the shape bound by impassable terrain. Medial
axis can be understood as skeleton of a shape. There are two main
categories of algorithms obtaining medial axis - skeletonisation and
thinning [8].
Skeletonisation algorithms aim to obtain a set of points which are
equidistant from nearest boundaries. To find these points, all map tiles
would get assigned their distance from the nearest edge and finding
skeleton would become a task of finding all locally maximal tiles. This
approach has been tried, but resulting skeletons were often disjointed
and tremendous effort had to be made to piece them together [8].
Thinning consists of examining immediate neighbourhood of any
given point in order to identify pixels that are, essentially, redundant
to the shape, i.e. those lying on and edge of a circle. By removing
such points, we are able to obtain an approximation of medial axis,
insensitive to boundary noise. Compared to skeletonisation, thinning
always results in fully connected shape, and lacking the extra join-
ing overhead, performs faster. Needless to say, it was chosen as the
preferred algorithm for this stage [8].
The map itself is traversed tile by tile, placing 1s on reachable tiles
and 0s on unreachable ones. Exact definition of reachable depends
132. TenguAI
on overall map topology. Potentially, the result can contain several
disjointed areas of reachable space, representing continents and/or
different elevation levels unreachable by preferred propulsion system.
Interpreting this two-dimensional array as a binary image, a thinning
algorithm is used to obtain medial axis.
Figure 2.1: ’Picturisation’ of a map. Minimap (left) compared binary
picture (right). Red denotes impassable tiles.
Initially, two identical copies of the image are created. One of the
is declared current and thinning iteration commences. For each tile, if
a thinning condition is met, it is set to 0 (non-shape). This condition
is evaluated on the other image, as changes done to the image may
interfere with the algorithm. Referring to table 2.1, the condition is as
follows:
Table 2.1: Visualisation of tile neighbourhood [8]
P3 P2 P9
P4 P1 P8
P5 P6 P7
142. TenguAI
2 ≤ NZ ( P1 ) ≤ 6
∧
Z0 ( P1 ) = 1
∧
P2 · P4 · P8 = 0 ∨ Z0 ( P2 ) ̸= 1
∧
P2 · P4 · P6 = 0 ∨ Z0 ( P4 ) ̸= 1
where Z0 ( P1 ) is a number of zero to non-zero transitions on a path
P2 → P3 → · · · → P9 → P2 , and NZ ( P1 ) is the number of non-
zero neighbours of P1 [8]. In addition to this, one more condition is
applied to except POIs and keep them within the shape regardless.
This ensures natural formation of paths that connect them to the rest
of thinned shape.
Figure 2.2: Comparison of binary image (left) to its thinned counter-
part (right).
After whole image has been traversed, content of the modified
image is copied into the helper image and process starts again, until
no further change occurs in the image. At that point, the array is
scanned one more time. All tiles that have more than three non-zero
neighbours are declared as nodes. This results in clusters of nodes
being created at locations of crossroads, which is going to be handled
by the next step.
152. TenguAI
2.1.3 Graph creation
In this step, obtained thinned shape is used to establish edges between
nodes of the graph.
Figure 2.3: Thinned shape with nodes. Starting positions (green),
resources (blue), crossroads (brown).
Queue of nodes to be processed is established, initially containing
only node located at AI‘s starting position. From here, paths visible in
thinned image are traversed in breadth first manner. Visited tiles are
marked with ’-’ to prevent backtracking and potential cycle. If another
node is encountered, it is pushed into queue of nodes to be processed,
bidirectional edge is established between them and further traversal in
this direction is stopped. This stop is trivial, because paths are always
single pixel wide.
Now, the algorithm gets to deal with the redundant nodes present
in all crossroads. To preserve edges and graph integrity, these nodes
are merged together. Merging here meaning all edges from node A
are added into node B. No other attributes are affected and node A is
discarded afterwards. This is to be done until a single node remains
in the cluster.
162. TenguAI
Starting again at the node representing AI‘s base, the graph is
traversed via BFS. All connected nodes within manhattan distance of
1 are added into the cluster and searched for more nodes to add. After
no more nodes can be added, the cluster is scanned for POI nodes.
If any POIs were found within the cluster, all mundane nodes
directly connected to any POI nodes are merged into all POI nodes
they are connected to, repeating until cluster contains only POI nodes.
If no POIs were present in the cluster, nodes from both sides of
the cluster are merged into their immediate neighbour. As clusters
are stored in arrays, the first and last nodes are merged into second
and one-before-last nodes, respectively. This results in a single node
roughly at the center of the cluster.
At this point, all nodes that are left edgeless are removed. These
are unreachable and could interfere with graph usage.
Figure 2.4: Final graph of the map. Starting positions (green), re-
sources (blue), chokepoints (red), mundane nodes(black).
172. TenguAI
Upon close examination on some maps, edges can be found that
seem to skip over a node. These are obviously unwanted and need
to be removed, lest they interfere with pathfinding. These edges are
identified by forming an obtuse triangle with smaller angles being
less than 10 degrees. Algorithm identifies such triangles and removes
the edge matching onto the longest edge of said triangle.
As the last step, if the map topology is not ’land’, edges are assigned
’water’ property if they connect nodes divided by a body of water.
These are identified by a localised version of thinning. Whole process,
as described above, is applied to a minuscule section of the map. This
section is a rectangle containing nodes of the edge in its opposing
corners. Water is set as 0s, everything else as 1s. Thinning is then
applied and if the nodes are not connected by the medial axis, edge is
declared ’water’ and will be only pathed through for water capable
propulsions.
2.1.4 Graph representation and manipulation
With map analysed and graph created, its actual structure and use
needs to be discussed.
Nodes themselves contain information pertaining to the general
area surrounding the node. Besides type of the node and edges, as
discussed above, nodes keep track other strategically important infor-
mation.
LastSeen timestamp is used to send out scouting parties to obtain
information about enemy presence (or lack thereof) around given
coordinates. This information consists of the following: fortification,
describing how much is the area fortified; owner, denoting last known
controller of the area; chokepoint flag, identifying the node as well
defendable; and hazard level, the size of enemy force known to be
around the point.
References to nodes themselves are kept in two objects. One is in
two dimensional array on node‘s respective coordinates. This array is
used to quickly fetch any given node at or nearby any given coordinates.
Second one is in simple array, which is used for iteration over all nodes.
Units, structures and even terrain features almost never correspond
to exact node positions. Ergo, an algorithm has been devised to match
any given coordinate to the nearest node. It works simply as follows:
182. TenguAI
starting at given coordinates, if node is present, return it. Else examine
tiles 1 away from original point. This examines an edge of progres-
sively larger rectangular portion of 2D array containing nodes, up to
a distance equal to half of longer edge of the map. This limit ensures a
node is always found.
For pathfinding through the graph, two algorithms are used, based
on information required. Any time the AI needs to find nearest x or
path from A to B, these algorithms are used.
For finding nearest x nodes satisfying a condition, a modified
breadth first search (further as BFS) is used. In its base form, BFS
is an uninformed (meaning no further information other than the
graph itself is provided) search strategy that examines all immediate
neighbours of a given node (regardless of distance), then neighbours
of these neighbours and so on, until a match is found or it runs out
of nodes [9]. First modification is the ability to look for nodes based
on arbitrary conditions. Every time a node is processed, a predicate
function is run to determine if the node is to be returned. Second
modification is the option to return multiple matches instead of just
a single node. This is done by filling an array with viable nodes until
either a given amount is reached or the algorithm runs out of nodes.
Third alteration is the implementation of a priority queue, making the
search prefer to expand closer nodes.
For pathfinding, AI uses A* algorithm. A* is by nature an informed
search. This additional information is provided via heuristic function
that alters preference values for every node. For every node to be
expanded, A* computes the cost of expanding each node, in this case
distance from source node, adds value given by heuristic function
and then expands the cheapest node available. This is repeated until
target node is reached and path taken to reach this node is minimal, as
long as heuristic function satisfies two conditions: admissibility and
consistency [9].
Admissibility means heuristic function will never evaluate cost to
target node as greater than what it actually is. Example of this would
be straight line distance, which AI partially uses. Consistent function
adheres to triangle inequality - estimated cost from node A to target
is not greater than cost to neighbour node B plus estimated cost from
B to target.
192. TenguAI
As a heuristic, this AI uses an addition of straight line distance and
hazard level of a node. This is to ensure that the safest path is found
first, if the shortest one would result in unnecessary danger to units.
Periodically, AI scans area around all nodes for enemy contacts. If
it finds anything, it saves the position and composition of enemy force
or base located. As groups of enemy units tend to move around the
map quite often, it also attempts to compute whether it has seen the
group before. This is done by the following algorithm:
1. Note down current information about the group: owner, compo-
sition, current game time and map position;
2. Based on components of a single randomly chosen unit, compute
its movements speed.
3. For each group G, compute speed required to move from posi-
tion of original group to G‘s location in amount of time elapsed
from the time G was last seen.
4. If this required speed is less than or equal to group‘s speed AND
G is closest such group found, set G as most likely match.
5. If a group satisfying these conditions was found, update its info
with current group.
Base location is much easier. Once a structure was revealed, it
stays visible to the player until they explore the area again. With this
knowledge, the AI simply scans the nodes for any visible structures
present in the vicinity and notes down their location, amount and
owner.
2.2 Cores
As with every RTS game, decision making process in Warzone 2100
can be divided into two separate, mutually supportive categories:
macromanagement and micromanagement. For both of these, separate
controller units - cores - were created in order to separate different
aspects of play.
202. TenguAI
2.2.1 Macromanagement cores
Macromanagement consists decisions affecting the flow of the game
on global scope. Strategy, unit production, resource procurement, re-
search and development all fall under this category. In the case of
Warzone 2100, unit design is also considered part of macromanage-
ment. Each of these aspects is represented by a core.
Main core
This core is the heart of macromanagement decision-making process.
It contains a neural network trained to output preferred strategy based
on current and previous game state. This is further elaborated on in
section 2.3.
There are three strategies available to AI, based on what field it
should try and gain/maintain advantage over the opponent. These
strategies being production, defense and technology. Focusing on
one of these, the AI changes processing order of other macro cores,
essentially assigning them priority access to resource stockpile.
With production focus, the AI attempts to gain numerical superi-
ority.
When focusing on defense, the AI fortifies its resource extractors
and base/entrance to the base, if available. Research is preferring
defensive structures and upgrades to their longevity.
When attempting to gain technological advantage, the AI gives
more value to researching new components and upgrades to unit
health, armor and damage.
Engineering, Production and Research cores
This trinity of cores represent basic strategic aspects common across
most RTS games. The all include an object mapping each structure/unit
type/research item to its perceived value to the AI.
Communication between these cores is done via requests. When
a core finds out it needs something falling under the jurisdiction of
another core, it send a request stating what it needs. The other core
than increments the value of requested item by an amount defined
by the main core. This amount is defined by relevance of said item to
212. TenguAI
the current plan. Any time the AI commits to a plan this value gets
incremented by 1.
Research and production cores are fairly simple both in their pur-
pose and implementation. They manage their respective facilities and
queue up items for research/production based on their perceived
worth.
Engineering core serves as hub for constructor units and manages
the construction of base structures and defensive works.
Constructor units are divided into 2 categories - main group and
roamers. Main group stays within base perimeter and does most of the
constructing. Roamers are dedicated to building resource extractors,
progressively further away from the base. This results in rather ag-
gressive early game expansion, very similar to Bonecrusher‘s (barring
the defenses), and is deemed vital to establishing healthy resource
income.
In addition to structures having priority values, engineering core
also keeps track of sites that have been requested to be fortified, again
with priorities. From these, every time the AI decides to build defenses,
the most pressing location is picked.
Design core
Design of units is one of the defining aspects of Warzone 2100, and as
such, has its own dedicated core. It keeps unit templates and sets of
components available for use.
The goal of the design process is to create a unit which deals maxi-
mum amount of damage to the opponent‘s army while simultaneously
minimising damage it takes from the same force. Due to nature of
the components, however, it is impossible to create a super-unit that
would be good against everything. For this reason, the designer core
creates several templates which focus on different types of enemies.
The composition of these different units into squads is left for other
cores to handle.
Templates themselves are objects containing components making
up the unit and its final properties, like damage, speed and health.
The API does provide its own templates, but these were deemed
insufficient as they lack most attributes required for proper design
process.
222. TenguAI
Both droid and cyborg templates have three variants, depending
on their preferred targets: anti-tank, effective against droids, preferring
to wield missiles and cannons; anti-personnel against cyborgs with
machine guns and flamethrowers and anti-structure using mortars
and howitzers to deal with enemy emplacements.
Every weapon in game has three attributes defining its purpose:
impact class, impact type and damage modifier. These are usually
bound together, but not always, which can cause some issues.
Impact type is the type of damage a unit deals. It can be either heat,
as in case of flamethrowers, which is mitigated by thermal armour, or
kinetic, which can be prevented by simple armour.
Impact class, or upgrade class for clarity, is the class of research
upgrades the weapon benefits from. This does not take into account
what type of damage the unit actually deals and improves all weapons
in the class equally.
Damage modifier is what actually matters when trying to maximise
damage. All damage dealt to a target is multiplied by this value, which
depends on propulsion or structure hardness any given target has. As
mentioned, this does not always correspond to the impact class. Most
notable case being class of rockets: while all benefiting from same
research, some rocket weapons are considered anti-tank in the terms
of their damage modifier, while others are marked as artillery.
Majority of Warzone 2100‘s strategy is in army composition and
ensuring you can destroy both enemy droids and cyborgs quickly,
and diluting one‘s research with weapons of class one does not plan
on upgrading, or upgrading weapons one does not use, is not only
pointless but detrimental. Thus, the AI picks three impact classes
to pursue, one for each unit type and one as anti-structure option.
It does this by sorting all weapons possible to research into arrays
corresponding to different impact classes. From these, one class per
purpose is chosen at random, mainly to maintain variety in game style.
As a sign of decency, the actual design process is not started if the
Command Center has not been built. This is to adhere to the rule of
human players being unable to open the designer window under the
same circumstances, which AIs are not inherently bound to.
Designer core keeps a template of the most numerous enemy unit
encountered so far and bases its design choices on this template.
Droids are designed by the following process:
232. TenguAI
1. From available bodies, such is chosen as to improve both health
and armor/thermal armor (depending on enemies weapons);
2. From available propulsions, one is chosen to minimize damage
taken from enemy weapons. In case the map is not universally
reachable by land units, purely land-based propulsions are ig-
nored;
3. From available turrets, turret capacity of chosen body id filled
with a weapon with greatest damage against most common
target.
For VTOLs, propulsion is pre-determined to be VTOL and so this
step is skipped. For cyborgs, only weapon is being chosen, as they are
hard-coded by the game to only take one specific propulsion type and
one of two body types, hard-linked to the weapon.
After design of a unit type is complete, designer sends research
requests for new components, based on the same logic as stated above.
Upgrades are also requested, with weapon upgrades being limited to
chosen upgrade paths.
2.2.2 Micromanagement cores
Micromanagement is the process of controlling individual units or
groups of units, in order to achieve strategic objectives (i.e. destroy
enemy base). Unit movement, their placement during combat and
flanking all belong into this category. Cores responsible for microman-
agement are divided based on the amount of units they control.
Sergeant core
This is the basic formation of AI‘s army. It contains up to predeter-
mined amount of units of specific type and using the same propulsion.
This last condition was added in order to streamline the pathfinding
process and secure that all units under single sergeant are able to reach
their target.
When the core takes fire from an enemy unit, a response rou-
tine starts. First, the core looks around and obtains total amount of
friendly and hostile units within certain radius. If the enemy has more
242. TenguAI
units present, the core sends a reinforcement request to its superior. If
granted, the core stays in combat, otherwise it attempts to retreat.
Sergeant cores can be in one of 4 states - idle, moving, engaged
and retreat.
When idle, the core is willing to accept new assignments from
superior core. If it detects that combined health of its units dropped
below predetermined percentage, it attempts to move to nearest repair
facility for repairs.
When moving, the core is in transition towards given coordinates
to perform assigned task. This task can be one of:
∙ attack - the core moves towards location actively engaging any-
thing it meets;
∙ scout - the core moves towards given coordinates ignoring ene-
mies, unless it is engaged first;
∙ hold - the core is sitting on top of a location, guarding against
enemy attacks;
∙ retreat - the core requests a repair facility to be built and attempts
to locate either repair facility or nearest friendly and fortified
position and moves there to regroup. If no such position is found,
it returns back to base;
∙ follow path - core carefully follows given path, acting similarly
to scout order;
∙ disband - core moves all its units to nearest factory for recycling.
Once in combat, the core locates, for each unit, the nearest enemy
said unit can effectively engage. Then, it orders the unit to attack and
carefully moves it around to stay at the edge of its short weapons
range. Within this range, the unit benefits from increased accuracy,
and thus, higher damage potential.
When retreating, the core is in panic and ignores all orders and
attacks until its destination is reached.
Any time a sergeant is completely wiped out during battle, it asks
superior core to reevaluate its current unit type and assign a new one,
based on current situation.
252. TenguAI
Colonel core
Colonel core is the central hub for micromanagement. It coordinates
sergeants and makes tactical decisions based on known enemy strength
and location.
This core‘s process can be divided into two main parts: threat
evaluation and planning.
Threat evaluation follows the following pattern:
1. All known enemy groups are assigned a threat value defined as
groupSize/distanceFromOwnBase;
2. If the enemy has (globally) more cyborgs than droids, produc-
tion priority of anti-personnel units is incremented. Same hap-
pens for anti-tank units in case of prevalence of droids;
3. If the enemy has in total more units that AI‘s sergeants can
currently manage, add a new sergeant of type most effective
against most numerous enemy and weapon type;
4. If the most threatening group has overwhelming numbers (more
than double of AI‘s total), find a path from the group‘s last
known location and request fortification of chokepoints on this
path;
5. Find path from AI‘s base to the base of focused enemy. If the
enemy has more fortifications on any given node on this path
than AI‘s anti-structure unit count, add new sergeant of anti-
structure role.
During planning, if on disjoint map, the AI tries to identify if it
has any enemies left on its continent. If not, it instruct the designer
to favour water-capable propulsions and orders the disbanding of
all land-based sergeants, if an alternative, water-crossing design is
available.
Additionally, if the macro plan is to focus on defense, the AI re-
quests fortifications to be built in its base, the entrance to its base, if
available, and all currently controlled resource locations. Then, threats
are ordered by their severity and individual idling sergeants are or-
dered around depending on currently selected micro plan. This plan
can be one of attack, skirmish, de f end.
262. TenguAI
If the plan is to attack, the AI tries to engage strongest known
enemy group or assault enemy base, if no such group is known. If
a group is to be engaged, the AI first attempts to muster strong enough
force to have a chance of victory. This is done by pooling sergeants
together, until their combined unit count is at least that of the targeted
enemy group.
In case of skirmishing, the AI looks over nearest 5 known enemy
derricks, picks one with weakest know enemy presence and pools
sergeants together, until their strength is greater than perceived hazard
around targeted derrick. If no derrick is found, it sends the sergeant
against known, weak enemy group; condition being the sergeant must
outnumber it at least 2:1. If all fails, the sergeant is sent to scout nodes
that have not been seen for a considerable amount of time.
When defending, the sergeant is sent to hold a fortified position
lying on a shortest path that the most threatening enemy group can
take to AI‘s base. If no such node exists, the sergeant holds entrance
to the base, if it exists, else it loiters around the base itself.
With orders prepared, colonel actually gives orders to sergeants
who are in groups satisfying strength requirements for their targets.
Any remaining sergeants are ordered to hold either entrance to base
or base itself.
2.3 Neural network for playstyle selection
Neural networks are machine learning systems loosely based on the
function of biological brain. Every such network contains layers of
discrete units - neurons. Outputs from all neurons in any given layer
are used as inputs for successive layer [10].
Every neuron takes a vector of weighted, real-valued inputs, sums
them and outputs a result of application of its activation function on
this sum. All neurons in a network have the same activation function
[10].
Goal of the learning process is to determine values for all input
weights, such that the final outputs match desired values [10].
272. TenguAI
For the purposes of this AI, it was decided ConvNetJS1 library was
to be used. This has a rather simple reason - it was the only ECMA 5.1
compatible neural network library I was able to find.
ConvNetJS offers several options when training neural networks.
Of these, we are interested in method, activation function and L2 decay.
Method specifies algorithm to use for automatic adjustment of the
learning rate of the trained network. L2 decay represents how tightly is
the network fitted onto training data. Generally, high L2 decay results
in very general networks, often to detriment of their accuracy.[11]
The training of this network was done on a set of 727 games played
against several different AIs under different conditions on multiple
maps, all further detailed in appendix A. Opponents were placed on
a random starting position. During each game, following procedure
was followed:
1. Pick a random plan from the list. Save current game state infor-
mation.
2. After approximately 30 seconds, evaluate current game state and
compare it to previous one. If an advantage in field chosen in step
1 was attained or improved, output previous state information
with the plan ID. Else, pick a different plan at random.
The idea behind this is - if the AI is able to maintain an advantage
throughout the course of a match, it should result in a victory.
Data from these games, in total 6701 entries, was divided into
training and test sets by an 80/20 split. The training set was then fed
into and evaluator script in an attempt to obtain the best parameters for
final network. For methods, ’Adadelta’ and ’Adagrad’ were evaluated
in combination with following:
∙ One of sigmoid, tanh and ReLU 2 (rectifier) for activation func-
tion;
∙ L2 decay of values 0.1, 0.05, 0.01, 0.005, . . . , 0.0001;
∙ Up to three neuron layers with number of nodes in range [5, 50]
in increments of 5.
1. https://cs.stanford.edu/people/karpathy/convnetjs/index.html
2. http://proceedings.mlr.press/v15/glorot11a/glorot11a.pdf
282. TenguAI
Within this script, training data were further split into training and
evaluation sets, again by 80/20 ratio.
Out of these options, the best accuracy was achieved by Adadelta3
with rectifier activation function on 0.0001 L2 decay with 2 neuron
layers of 40 and 5 neurons, respectively. Its accuracy was 88.71 % and
thus these settings were used to train the final neural network. This
network was tested on the yet unseen 20 % of data and its final accuracy
was measured at 66 %.
Then, another set of 558 games has been run, this time with the
network managing macro plan selection. The results can be seen in
table B.2, in comparison with pre-network statistics (table 2.2). Draw
result was declared when the match was not over after two hours of
playtime.
Table 2.2: Match statistics during learning process, per opponent.
NullBot Bonecrusher Cobra Total
Win 119 73 75 267
Loss 95 155 155 405
Draw 20 24 11 55
Win Rate 50.85 % 28.96 % 31.12 % 36.72 %
Table 2.3: Match statistics with learned neural network, per opponent.
NullBot Bonecrusher Cobra Total
Win 88 49 54 191
Loss 89 101 122 312
Draw 19 32 4 55
Win Rate 44.90 % 26.92 % 30.00 % 34.23 %
From this data, it appears the inclusion of neural network had no
to slightly detrimental effect on the AIs performance. This may be
due to low accuracy of the network or poorly chosen variables chosen
as input. However, seeing the win rate dropped by 2.49 %, it may
3. https://www.matthewzeiler.com/mattzeiler/adadelta.pdf
292. TenguAI also very well be simply circumstantial occurence - opponents simply happened to play better in this set of games. Regardless of the cause, success of the AI is mediocre at best. 30
3 Conclusion
After all is said and done, TenguAI comes out as mediocre to subpar
in terms of its competitive ability. However, the working implementa-
tions of both map analysis and neural network open doors for more
developers to pick up the proverbial torch and include these sub-
systems in designs of their own, hopefully opening new door in AI
capability within Warzone 2100.
With regard to TenguAI and its possible extensions, more work
can definitely be done on its micromanagment subsystems, target
acquisition and sergeant coordination. It may also be worthwhile
endeavor to implement second neural network, focused on selection
of micromanagement plans.
Another option to enhance the AI would be to develop a multi
criteria optimisation system for unit designer, making priority list of
attributes and designing units based on that.
Finally, the AI was released to the community under CC0 license.
31A Maps used for neural network training
Note: all pictures present are screenshots of minimaps within the game.
Figure A.1: Small map with tight corridors and perfect early game
chokepoint in the middle.
Figure A.2: Water-separated continents with varying amounts of play-
ers. Ideal for testing analyser and cross-water attacks of the AI.
33A. Maps used for neural network training Figure A.3: Sizeable, but tight map with hill-separated pathways. Good for concentrated defenses and macro movement. Figure A.4: Small but spacious map. Good for quick matches that do not require fancy movements. 34
A. Maps used for neural network training
Figure A.5: Personal favourite. Tight, highly asymmetric starting posi-
tions.
Figure A.6: Similar in concept to Monocot, but with more resources
and corridors.
35B Statistics of matches before and after the use
of neural network
Table B.1: Match statistics during learning process, per map and game
setting.
Monocot Fish Nets Manhattan Miza Maze
Win 59 48 39 35
Loss 102 78 49 66
Draw 1 7 20 7
Win Rate 36.42 % 36.09 % 36.11 % 32.41 %
Mountain Rush Scavengers No Scavengers
Win 31 55 146 121
Loss 59 51 184 221
Draw 18 2 33 22
Win Rate 28.70 % 50.93 % 40.22 % 33.24 %
37B. Statistics of matches before and after the use of neural network
Table B.2: Match statistics with learned neural network, per map and
game setting.
Monocot Fish Nets Manhattan Miza Maze
Win 19 24 37 45
Loss 37 42 46 58
Draw 0 4 25 5
Win Rate 33.93 % 34.29 % 36.11 % 34.26 %
Mountain Rush Scavengers No Scavengers
Win 29 37 93 98
Loss 58 71 151 161
Draw 21 0 35 20
Win Rate 26.85 % 34.26 % 33.33 % 35.13 %
38Bibliography
1. WZ2100 Guide: Early-game multiplayer strategy [online] [visited
on 2019-11-11]. Available from: http : / / betaguide . wz2100 .
net/earlygamestrategy.html.
2. Introduction to Warzone 2100 [online] [visited on 2019-11-11].
Available from: http://betaguide.wz2100.net/Introduction.
html#.
3. A History of Real-time Strategy Games [online] [visited on
2019-11-08]. Available from: https://web.archive.org/web/
20110427052656/http://gamespot.com/gamespot/features/
all/real_time/.
4. WZ2100 Guide: Research tree [online] [visited on 2019-11-11].
Available from: http://betaguide.wz2100.net/Research.php?
tree=1.
5. WZ2100 Guide: Weapon modifiers [online] [visited on 2019-11-11].
Available from: http://betaguide.wz2100.net/weapons.php?
damageModifiers=1.
6. Bonecrusher thread on WZ2100 forum [online] [visited on 2019-11-
14]. Available from: http://forums.wz2100.net/viewtopic.
php?f=49&t=12758.
7. MILLINGTON, Ian; FUNGE, John David. Artificial Intelligence for
Games. 2nd ed. Burlington: Morgan Kaufmann Publishers, 2009.
ISBN 978-0-12-374731-0.
8. JAIN, Anil K. Fundamentals of Digital Image Processing. Eaglewood
Cliffs: Prentice Hall, Inc., 1989. ISBN 0-13-336165-9.
9. RUSSEL, Stuart J.; NORVIG, Peter. Artificial Intelligence: a modern
approach. 3rd ed. Upper Saddle River: Prentice Hall, Inc., 2010.
ISBN 978-0-13-207148-2.
10. MITCHELL, Tom M. Machine Learning. Boston: McGraw-Hill,
1997. ISBN 0070428077.
11. ConvNetJS - Deep learning in your browser. Available also from:
https : / / cs . stanford . edu / people / karpathy / convnetjs /
docs.html.
39You can also read
