Analysis and management of EVTC logs - Jiˇr ı Sejkora

 
CONTINUE READING
Analysis and management of EVTC logs - Jiˇr ı Sejkora
BACHELOR THESIS

                          Jiřı́ Sejkora

Analysis and management of EVTC logs

         Department of Distributed and Dependable Systems

  Supervisor of the bachelor thesis: Mgr. Pavel Ježek, Ph.D.
                Study programme: Computer Science
                     Study branch: General Computer Science

                             Prague 2021
Analysis and management of EVTC logs - Jiˇr ı Sejkora
I declare that I carried out this bachelor thesis independently, and only with the
cited sources, literature and other professional sources. It has not been used to
obtain another or the same degree.
I understand that my work relates to the rights and obligations under the Act
No. 121/2000 Sb., the Copyright Act, as amended, in particular the fact that the
Charles University has the right to conclude a license agreement on the use of this
work as a school work pursuant to Section 60 subsection 1 of the Copyright Act.

In . . . . . . . . . . . . . date . . . . . . . . . . . . .       .....................................
                                                                            Author’s signature

                                                              i
Analysis and management of EVTC logs - Jiˇr ı Sejkora
I would like to thank my supervisor Mgr. Pavel Ježek, Ph.D for his guidance,
patience, and useful advice. I would also like to thank my family and all the
friends who have supported me throughout my studies and during writing this
thesis.

                                     ii
Analysis and management of EVTC logs - Jiˇr ı Sejkora
Title: Analysis and management of EVTC logs

Author: Jiřı́ Sejkora

Department: Department of Distributed and Dependable Systems

Supervisor: Mgr. Pavel Ježek, Ph.D., Department of Distributed and Dependable
Systems

Abstract: EVTC is a binary format of combat logs generated by arcdps, an addon
for Guild Wars 2, a multiplayer MMORPG game. These combat logs are used to
analyze combat performance, which allows players to improve their gameplay.

Using the .NET platform and the Eto.Forms framework, we have built a cross-
platform program for managing EVTC logs. The program allows players to ef-
ficiently browse through thousands of EVTC logs, search for specific logs and
upload them to third-party services.

We have also created a new .NET library for extracting and interpreting data
from EVTC logs. This library is used for the log manager and may be used
for other projects in the future. In the process, we have discovered new reliable
methods of extracting high-level results from EVTC logs.

Keywords: EVTC video game combat logs .NET MMORPG

                                       iii
Analysis and management of EVTC logs - Jiˇr ı Sejkora
Contents
I     Introduction                                                                                                                 4
1 Introduction                                                                                                                     5
  1.1 Combat logs . . . . . . . . . . . . . . . . . . . . . . . . .                                           .   .   .   .   .    5
  1.2 Guild Wars 2 . . . . . . . . . . . . . . . . . . . . . . . .                                            .   .   .   .   .    5
  1.3 arcdps . . . . . . . . . . . . . . . . . . . . . . . . . . . .                                          .   .   .   .   .    6
  1.4 EVTC logs . . . . . . . . . . . . . . . . . . . . . . . . .                                             .   .   .   .   .    7
  1.5 Existing EVTC log data extraction implementations . . .                                                 .   .   .   .   .    8
       1.5.1 GW2Raidar . . . . . . . . . . . . . . . . . . . . .                                              .   .   .   .   .    8
       1.5.2 L0G-101086 . . . . . . . . . . . . . . . . . . . . .                                             .   .   .   .   .    8
       1.5.3 Elite Insights . . . . . . . . . . . . . . . . . . . .                                           .   .   .   .   .    8
       1.5.4 Motivation for implementing a new EVTC library                                                   .   .   .   .   .   10
  1.6 The dps.report service . . . . . . . . . . . . . . . . . . .                                            .   .   .   .   .   10
  1.7 Choice of technology . . . . . . . . . . . . . . . . . . . .                                            .   .   .   .   .   10
  1.8 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                                         .   .   .   .   .   11

II    EVTC Logs                                                                                                                   12
2 EVTC                                                                                                                            13
  2.1 Concepts . . . . . . . .    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   13
      2.1.1 Agents . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   13
      2.1.2 Skills . . . . . .    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   15
      2.1.3 Movement . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   15
  2.2 Log contents . . . . . .    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   16
      2.2.1 Header . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   16
      2.2.2 Agents . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   16
      2.2.3 Skills . . . . . .    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   17
      2.2.4 Combat Items .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   17
  2.3 Limitations . . . . . .     .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   18
      2.3.1 Reporting range       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   20
      2.3.2 Missing damage        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   20
      2.3.3 Conditions . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   20
      2.3.4 Healing . . . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   21
      2.3.5 Buff extensions       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   21
  2.4 Malformed logs . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   22

III    Design                                                                                                                     23
3 EVTC library                                                                                                                    24
  3.1 Data representation . . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   24
  3.2 Step 1: Parsing . . . . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   24
      3.2.1 Revision compatibility                .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   24
      3.2.2 Data container . . . .                .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   24
      3.2.3 Editing . . . . . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   25

                                                  1
Analysis and management of EVTC logs - Jiˇr ı Sejkora
3.3    Step 2: Processing . . . . . . . .                 .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   25
          3.3.1 Relational data . . . . . .                  .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   25
          3.3.2 Encounter data definitions                   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   25
          3.3.3 Post-processing steps . . .                  .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   26
   3.4    Step 3: Analysis . . . . . . . . . .               .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   26
   3.5    Testing . . . . . . . . . . . . . . .              .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   27

4 Log     Manager                                                                                                                    28
  4.1     Choosing a GUI framework .                 .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   28
  4.2     User interface . . . . . . . . .           .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   30
  4.3     Background services . . . . .              .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   33
  4.4     Extracting data from logs . .              .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   34
          4.4.1 Elite Insights . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   34
          4.4.2 A new EVTC library .                 .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   34
          4.4.3 Log discovery . . . . .              .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   34
          4.4.4 Processing . . . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   34
          4.4.5 Log data caching . . .               .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   35
          4.4.6 Log processing updates               .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   36
   4.5    Settings . . . . . . . . . . . .           .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   36
   4.6    Guild Wars 2 API . . . . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   37
   4.7    Uploads to dps.report . . . . .            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   38
   4.8    Statistics . . . . . . . . . . . .         .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   38
          4.8.1 Player statistics . . . .            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   38
          4.8.2 Guild statistics . . . .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   38
   4.9    Update checking . . . . . . .              .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   39
   4.10   Data architecture . . . . . . .            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   39

IV       Implementation                                                                                                              40
5 Implementation overview                                                                                                            41
  5.1 Project structure . . . . . . . . . . . . . . . . . . . . . . . . . . .                                                        41
  5.2 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .                                                      42

6 EVTC library                                                                                                                       43
  6.1 Encounter detection . .            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   43
  6.2 Encounter data . . . . .           .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   43
      6.2.1 Success detection            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   43
      6.2.2 Mode detection .             .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   46

7 Log     Manager                                                                                                                    49
  7.1     The life cycle . . . . .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   49
  7.2     The Manager Form . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   49
  7.3     Dependency Injection .     .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   50
  7.4     Settings . . . . . . . .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   50
  7.5     User interface layouts .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   51
  7.6     User interface sections    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   51
  7.7     User interface controls    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   52
  7.8     Settings user interface    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   53

                                                     2
Analysis and management of EVTC logs - Jiˇr ı Sejkora
7.9 Debug data . . . . . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   53
    7.10 Log discovery . . . . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   54
         7.10.1 Directory monitoring        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   54
    7.11 Filters . . . . . . . . . . . .    .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   55
    7.12 Open-source contributions .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   55
    7.13 Distribution . . . . . . . . .     .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   56

V     User Documentation                                                                                                        58
8 EVTC library                                                                                                                  59

9 Log      Manager                                                                                                              60
  9.1      Running the Log Manager . . .            .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   60
  9.2      First launch . . . . . . . . . . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   60
  9.3      User interface overview . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   61
  9.4      Filtering . . . . . . . . . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   62
  9.5      Log list . . . . . . . . . . . . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   64
           9.5.1 Table columns . . . . . .          .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   64
           9.5.2 Details pane . . . . . . .         .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   65
           9.5.3 Sorting . . . . . . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   66
    9.6    Player list . . . . . . . . . . . .      .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   66
    9.7    Guild list . . . . . . . . . . . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   67
    9.8    Statistics . . . . . . . . . . . . .     .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   68
    9.9    Menu options . . . . . . . . . .         .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   69
    9.10   Status bar . . . . . . . . . . . .       .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   70
    9.11   Configuration . . . . . . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   71
    9.12   Automatic discovery of new logs          .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   71
    9.13   Stored data . . . . . . . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   72
    9.14   Updates . . . . . . . . . . . . .        .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   72
    9.15   EVTC Inspector . . . . . . . .           .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   72

VI     Conclusion                                                                                                               74
10 Conclusion                                                                                                                   75

Conclusion                                                                                                                      75
  10.1 New methods for analyzing EVTC logs . . . . . . . . . . . . . . .                                                        77
  10.2 Log Manager releases . . . . . . . . . . . . . . . . . . . . . . . . .                                                   77
  10.3 EVTC Inspector . . . . . . . . . . . . . . . . . . . . . . . . . . .                                                     77

11 Future work                                                                                                                  78

Bibliography                                                                                                                    80

A Attachments                                                                                                                   83

                                                3
Analysis and management of EVTC logs - Jiˇr ı Sejkora
Part I

Introduction

     4
Analysis and management of EVTC logs - Jiˇr ı Sejkora
1. Introduction
The world of multiplayer video games consists of many genres, in most of which
there is demand for gameplay optimization. This includes combat-focused games
where a player may want to maximize their damage output, and analytic tools
are often developed for this purpose.

1.1        Combat logs
Exporting combat logs that may be analyzed later is a common approach in this
area. A log file of a combat encounter is first produced by a game or an addon
capable of reading the state of the game. This file may then be interpreted at a
later time by software tools to allow players to analyze the encounter. Examples
of such tools include:

  • Warcraft Logs [1], a website analyzing and comparing combat logs gener-
    ated by World of Warcraft, an MMORPG1 game published by Blizzard
    Entertainment;

  • Advanced Combat Tracker [2], a general program for recording and analyz-
    ing logs from multiple games.

   This thesis focuses on interpreting combat logs in the EVTC format generated
by arcdps [3], a third-party addon for Guild Wars 2, an MMORPG game.

1.2        Guild Wars 2
Guild Wars 2 is a multiplayer online game developed by ArenaNet. The game was
first released in 2012, and is now available with two expansions: Heart of Thorns
released in 2015, and Path of Fire released in 2017.

Player versus Environment
Guild Wars 2 features multiple game modes. Players may explore a world consist-
ing of separate maps in which they battle computer-controlled enemies. Figure 1.1
shows an example of such combat and damage being inflicted to an enemy. This
type of combat is typically called PvE, which stands for Player versus Environ-
ment. The game notably contains instances with PvE content designed for 5 or
10 players cooperating together to defeat enemies.
    Optimizing gameplay in PvE instances has significant popularity; this is ev-
idenced by the leaderboards for Guild Wars 2 on speedrun.com [4], a website
recording fastest completions of games. Logs of combat encounters in PvE in-
stances are heavily used by the community to analyze and improve gameplay
efficiency.
  1
      MMORPG stands for Massively Multiplayer Online Role Playing Game.

                                            5
Analysis and management of EVTC logs - Jiˇr ı Sejkora
Figure 1.1: A screenshot from Guild Wars 2 showcasing PvE combat.

Player versus Player
Combat against players, known as PvP — Player versus Player, is not available
in any PvE maps, but is instead fought on maps designed for this purpose. There
are two modes of PvP combat available, however combat logs are only available
for one of these modes, World versus World. This mode — typically abbreviated
as WvW — is a mode in which 3 worlds compete for objectives in the form of
camps, towers and keeps, fighting in groups that may be over 50 players big.

1.3       arcdps
arcdps 1 is an addon for Guild Wars 2 developed by deltaconnected. The addon
shows realtime damage statistics (depicted in Figure 1.2) and produces combat
logs in the EVTC format.

Figure 1.2: A screenshot of the arcdps real-time widget showing damage of 10
players in an instance.

   1
    The author of arcdps rarely uses capital letters, which includes the name of the addon. We
respect this capitalization scheme throughout this text.

                                              6
arcdps versions
The arcdps addon has been available for multiple years and has gained extra
features over time. Throughout this text, we use a YYYYMMDD scheme used in
EVTC logs (detailed in Section 2.2.1) to refer to versions of arcdps. Version
YYYYMMDD corresponds to a version of arcdps released on day DD of month MM of
year YYYY. The author of arcdps generally aims to not release multiple versions
on the same day as they are indistinguishable.

arcdps limitations
The addon is implemented by reverse-engineering the game, and is loaded as
a proxy library, d3d9.dll. The addon heavily utilizes reading data stored in
memory of the game. Because of this mechanism, it is prone to issues when
the game is updated. The behavior with incompatible versions ranges from full
crashes of the game to minor imprecisions in its calculations.
    It is a common practice for multiplayer game servers to only transmit nec-
essary data for rendering the game to the game client. The addon reconstructs
what is happening within the game from its memory, and not all data necessary
for accurate simulation is available in some cases. The limitations of arcdps are
further detailed in chapter 2.

1.4     EVTC logs
Recorded EVTC logs are stored in a binary format unique to arcdps. A log
consists of basic metadata, content definitions further detailed in chapter 2, and
of combat items sequentially describing events within the encounter. It does not
contain any statistics or high-level outcomes, such as whether the encounter was
successful. These results have to be determined by external tools interpreting
EVTC logs.

Motivation for implementing a Log Manager
An EVTC log is stored either with an .evtc extension, or it may be compressed
as a ZIP archive, in which case logs use a .zevtc extension. The logs are stored
in directories named by the main target of the recorded encounter. Traditional
file managers offer no further support for EVTC logs, which makes it difficult to
efficiently work with recorded logs. It is not possible to find logs with specific
players, search only for successful logs, see durations of logs or other metadata.
Players commonly have tens of thousands of EVTC logs, with total file sizes
measured in gigabytes.
     We have identified a significant need for improvement in this area and we seek
to solve this by implementing a program for managing EVTC logs — an arcdps
Log Manager. In order to implement such a manager, we need to interpret EVTC
logs. In the following sections we research existing implementations of EVTC log
data extraction.

                                        7
1.5      Existing EVTC log data extraction imple-
         mentations
Many of our initial choices were made according to the state of the ecosystem
as of September 2018. However, the development is ongoing, and it has been
reacting to changes and new options as they become available. In this section, we
detail both the original state of tools in September 2018 and the current state,
as of May 2021.

1.5.1     GW2Raidar
GW2Raidar [5] is a now defunct website aggregating metrics from EVTC logs,
similarly to Warcraft Logs [1]. The website has been shutdown in July 2019, and
a later scheduled relaunch failed due to a lack of funding.
    The website is notable for its open-source parts which remain available, al-
though last contributions to the project were made in November 2019. The web-
site has used an open-source custom parser for EVTC logs written in Python 3.
The parser has not been updated for changes in the EVTC format and is unable
to correctly identify successful encounters after significant changes to rewards in
the game in June 2019.
    The GW2Raidar parser relies on reward events in EVTC logs, which are no
longer guaranteed to happen if a player repeats an encounter multiple times in a
week. This is a weakness shared with most simple EVTC parsers. Our analysis
of the source code has also identified significant bugs in its interpretation of raw
EVTC data.

1.5.2     L0G-101086
L0G-101086 [6] is a collection of PowerShell scripts used to upload EVTC logs
to services. The project includes simpleArcParse, an open-source C++ program
capable of parsing EVTC logs. The program has similar issues to the GW2Raidar
parser, lacking support for proper success detection, and only extracting limited
data.

1.5.3     Elite Insights
Elite Insights [7] is a parser of EVTC logs focused on generating HTML reports
with detailed statistics (depicted in Figure 1.3). Elite Insights is an open-source
.NET project written in C# available under the MIT license.
    Statistics of note include damage inflicted by each player, effect uptimes and
details of abilities used. High-level statistics, such as whether the encounter was
successful, its phases, failures to execute game mechanics properly, and others are
also available. The reports also include Combat Replay, an interactive animated
top-down replay of the encounter.
    We believe the area of generating such reports is in a good state with what
Elite Insights offers, and is not in need of significant improvements. Any such con-
tribution would be best aimed at improving Elite Insights instead of developing
a competing HTML report generator.

                                         8
Figure 1.3: A screenshot of the damage stats page of an HTML report generated
by Elite Insights.

State of Elite Insights in 2018
Elite Insights uses the Windows Forms framework for its user interface, but also
provides a command-line interface compatible with Mono, a cross-platform open-
source .NET Framework implementation.
    While some parts are optional (such as the combat replay), the codebase is
not modular and most classes are tightly coupled together. The project is not
available as a library and would require non-trivial changes in order to adapt
it so. Future versions may also significantly change the structure of the HTML
output, as it is not guaranteed to be stable.
    The application works with full raw event data everywhere, which is prone to
programming mistakes, as the knowledge of the full EVTC format is required in
all interactions with data from logs.
    The time spent processing a single log from start to finish was 2.5 seconds for
an average log after we have contributed significant performance improvements
[8, 9] to Elite Insights, which was still not sufficient to process tens of thousands
of logs, with an average runtime of 7 hours per 10000 logs. This performance was
not sufficient for use in a log manager.
    Because of the listed limitations, we have found Elite Insights to be unsuitable
for use in another project at this time.

                                         9
State of Elite Insights in 2021
As of May 2021, Elite Insights is the only (other than our own) up-to-date im-
plementation capable of correctly detecting successful encounters.
    Over the years, the code quality and performance has been significantly im-
proved. The core functionality is now available as a library that may be used
from other projects. The library is now capable of capable of providing our basic
data at an average rate of 5 logs per second.
    The entire project also no longer works directly with raw event data, instead
opting to process most events into type-safe objects first. This change has been
introduced in June 2019.

1.5.4     Motivation for implementing a new EVTC library
As we have established, there is no existing EVTC library suitable for use in a
log manager. In addition, to that, there are multiple other benefits from imple-
menting our own solution.

  • Performance tuning. We can optimize performance for our use case. This
    includes providing APIs that selectively calculate values that we need.

  • Opportunities for new research. Implementing this library includes imple-
    menting success detections and detecting encounter modes. We may find
    methods that are more reliable than the ones employed by Elite Insights.

  • Comparing with other implementations. We may use our library to verify
    the correctness of Elite Insights, and vice versa. This should result in higher
    quality for both projects.

  • Easy of use. The APIs of our new library may be easier to use for small
    projects, abstracting some internal details of the EVTC format.

1.6      The dps.report service
The dps.report [10] service is widely used by players to host Elite Insights reports
generated from uploaded EVTC files. The service provides a permanent URL
for each report that may be used to share reports with others. As of June 20,
2021, the website is hosting more than 7.72 million reports according to a counter
shown in the header of the page [11].
    As we seek to implement a program for log management, support for uploading
logs to this service is one of the main features it should provide.

1.7      Choice of technology
We choose to implement the Log Manager and the new library using the .NET
software framework and the C# language, using libraries with cross-platform sup-
port. At the very least, Windows 10 and common Linux-based operating systems
must be supported by the manager.

                                        10
The main reason this tooling choice is a significant prevalence of .NET use
by developers in the game community. There are multiple major open-source
projects written in C#, including Elite Insights, and accessibility for potential
contributors is important to us. The language also provides sufficient performance
and high quality tooling is available.

1.8      Goals
  1. Implement an open-source multi-platform program that will allow for easier
     EVTC log management — the arcdps Log Manager. This program has the
     following characteristics:

      (C1) The program is written in the C# programming language, using .NET
           frameworks.
      (C2) The program is available on Windows and Linux operating systems.
      (C3) The program is open-source and available for contributions.
      (C4) The program can effectively work with tens of thousands of logs.
      (C5) The program must handle any malformed logs without crashing.

      The log manager has to support the following features:

      (F1) Showing a list of logs, similarly to a file manager.
      (F2) Showing relevant logs details for selected logs.
      (F3) Filtering logs by their contents.
      (F4) Uploading logs to dps.report, and support batch uploads.
      (F5) Showing meaningful aggregated data while respecting filters.

  2. Implement a library an efficient and easy-to-use open-source .NET library
     for reading EVTC logs and extracting data from them, as a means to im-
     plement the first goal. The library needs to support the following features:

      (F1) Reading EVTC logs in both non-compressed and compressed versions.
      (F2) Identifying the encounter recorded by the log.
      (F3) Identifying players in the log.
      (F4) Recognizing successful encounters and failures.
      (F5) Recognizing encounter modes.
      (F6) Calculating the length of an encounter.
      (F7) Provide an API that allows access to raw data in EVTC logs.

                                        11
Part II

EVTC Logs

    12
2. EVTC
This chapter provides a high-level overview of the binary contents of an EVTC
log and the challenges involved in analyzing it.
    The mechanisms detailed in this chapter are partially abstracted away in later
chapters. However, knowledge of concepts in this chapter is useful in order to
understand some choices made in later chapters. This chapter is also useful to
understand limitations of the data available.
    The information in this chapter comes from the EVTC documentation [12]
as of arcdps version 20210104, from arcdps changelogs1 [3], and from original
research.

2.1       Concepts
The logs expose various internal representations of the game listed in this section.
Knowledge of the following terms is required to understand the contents of EVTC
logs.

2.1.1      Agents
An agent within an EVTC log is one of the following:
   • Player — an entity controlled by a human player,

   • NPC — a mobile entity,

   • Gadget — a static entity,

   • Attack target — a static entity that redirects damage to a gadget.
    Players are human-controlled entities and NPCs are computer-controlled en-
tities. Gadgets2 are a special type of an immobile entity with unique appearance
and targeting in the game. A gadget may have zero or more attack targets which
may switch states between targetable and untargetable. An attack target that
is not targetable cannot be hit by abilities. Figure 2.1 shows a gadget and an
attack target as it looks within the game.
    Attack targets are only useful for tracking their state of targetability. Events
affecting their parent gadgets, such as damage events, are attributed to the gad-
get.
    NPCs have a species ID, which is an ID of a template specifying the model
and skill set of an NPC. The same boss will always reliably have the same ID,
and species IDs are the main mechanism of identifying different encounters.
    The arcdps addon also assigns IDs to gadgets by hashing properties of the
gadget as an attempt at providing an equivalent to species IDs of NPCs. Gadget
IDs are not always reliable; gadgets which always appear in the same place in a
map may have a different ID in a different instance of the same map.
   1
      Old changelogs are no longer available on the arcdps website. We include a partial archive
of the changelog as an electronic attachment.
    2
      Also commonly called structures by players of the game.

                                              13
Figure 2.1: An attack target on a gadget.

    An NPC or Player may also have a secondary health bar — the defiance bar,
shown in Figure 2.2, commonly also referred to as a breakbar. This indicates they
are immune to crowd control effects, such as stuns, that would significantly impair
them and trivialize fights if repeatedly applied. Instead, these effects damage the
defiance bar and after fully depleting the defiance bar, varying effects may occur.
Many NPCs have attacks that require players to deplete the defiance bar to stop
them. Defiance bars are very common in instanced content on NPCs, but only
very rarely applied to players. Defiance bars have three states — active, locked
and recharging.

Figure 2.2: An NPC with a defiance bar in a locked state below the standard
health bar.

                                        14
Player NPC         Gadget         Attack target
                      Name            ✓     ✓            ✓         Encodes ID of master
         Maximum Health               ✓     ✓            ✓
                 Buff data            ✓     ✓            ✓
                Skill usage           ✓     ✓            ✓
              Damage data             ✓     ✓            ✓
               Hitbox size            ✓     ✓            ✓
   Master/minion relations            ✓     ✓            ✓
          Toughness (stat)           0–10   ✓            ✓
      Concentration (stat)           0–10   ✓            ✓
             Healing (stat)          0–10   ✓            ✓
   Condition damage (stat)           0–10   ✓            ✓
                    Events            ✓     ✓          Partial        Targetable state
             Master gadget                                                  ✓
          arcdps gadget ID                                ✓
                Species ID                      ✓
             Account name             ✓
            Group number              ✓
              Guild1 GUID             ✓

                     Table 2.1: Available data for all agent types.

   Table 2.1 lists available data and events for agents. Many events are not
reported for gadgets, such as changes of their combat state or deaths. This
makes analyzing fights involving gadgets more difficult.

2.1.2      Skills
A skill within the game is either an ability usable by an agent with various effects,
or a status effect that may be applied to an agent. Status effects are internally
referred to as buffs 2 in arcdps and its documentation regardless of their effects,
and we adopt this nomenclature as well. Similarly, abilities are often just referred
to as skills.
    Skills have various categories, effects and properties. This data is available
for skills present in the EVTC log since version 20191225. However, our under-
standing of the contents these definitions is limited as these definitions have not
been researched enough, and only the meaning of some effect ids is known.

2.1.3      Movement
The game is situated in 3D space, and as such, positions of agents have three
coordinates. Logs include position, velocity and facing data for agents since
version 20180626. In EVTC logs, the Z coordinate encodes height and is inverted
as lower values are higher up within the game world. The height of 0 is the static
water level which is present in all game maps, positive values are thus underwater.
   1
    A guild is a long-term group of players with a name and many features.
   2
    It is common to distinguish beneficial status effects – buffs, and detrimental status effects
– debuffs in MMORPG games.

                                               15
2.2        Log contents
The format is binary, with each item having a set size. The log starts with a
magic string of EVTC, and a header containing metadata. It is then followed by a
list of agents and a list of present skills. The last section is composed of individual
combat items describing events. Figure 2.3 illustrates the overall structure of the
file.
     Integral values vary in precision, they range from one byte to 8-byte integers.
For some combat items, multiple fields have to be merged and interpreted as a
value according to the specification. Strings, such as names, are encoded as UTF-
8 and have a buffer of a specified length padded with NUL1 bytes. The buffers
have sufficient space to avoid truncating long names.

                                    Header (16B)
                                 A = Agent count (4B)
                                   Agents (A × 96B)
                                 S = Skill count (4B)
                                   Skills (S × 68B)
                                  Combat item (64B)
                                  Combat item (64B)
                                          ..
                                           .
                                  Combat item (64B)

           Figure 2.3: The overall layout of the EVTC log as of revision 1.

2.2.1        Header
The log starts with a string of EVTCYYYYMMDD, where YYYYMMDD is a date which
indicates the version of arcdps used to generate this log. The next byte determines
the revision of the format. Revisions specify the binary format of further items.
The first revision was numbered 0 and the current revision is 1, which has brought
minor changes to the encoding of combat items.
   The last available value in the header is the ID of the target that has triggered
the log. This can either be an ID of an NPC species, an ID of a gadget, or 1 in
the case of a World versus World log. Finally, there is one unused byte at the
end of the header. Figure 2.4 illustrates the structure of the EVTC header.

2.2.2        Agents
The header is followed by a list of agents. This section starts with the count
of agent definitions that will follow. Each agent definition encodes the type of
  1
      Containing a value of 0.

                                          16
EVTC                             Version (8B)
        Revision (1B)             Target ID (2B)               Unused (1B)

                    Figure 2.4: The header of the EVTC log.

the agent, two distinct IDs used by arcdps (an instance ID and an address), and
values unique per agent type, such as the Profession and Elite Specialization of
a Player, or Species ID in the case of an NPC.
    A name is also saved, and in the case of non-player agents it will be local-
ized in the game language used by the recording player. For player agents, the
name contains their character name, their account name and their group number,
separated by NUL bytes.
    The definition of an agent also includes its hitbox size and some of their stats.
In the case of players, stats are stored as a value in range 0-10, where 10 stands for
highest of all players, 0 the minimum and other values are linearly interpolated.
This allows to identify some player gear choices, but it may also be misleading in
case the maximum stat amount of a player is small.

2.2.3     Skills
The list of agents is followed by a list of skills, which is encoded akin to agents. A
skill count is provided, and individual definitions follow. The definitions of skills
in this section only consist of their IDs and localized names. Further data related
to skill definitions is encoded within combat items since version 20191225.

2.2.4     Combat Items
Combat items form the rest of the log. Unlike the lists of agents and skills,
no count is provided in advance. Combat items both encode events that have
happened within the game and various metadata pertaining to the log. Most
combat items contain a system time timestamp which denotes when exactly they
happened.
   The encoding scheme differs drastically between various combat item types
and is detailed in the EVTC documentation [12]. They are categorized as follows:

  • Skill activation — triggered when an agent uses or finishes using a skill,

  • Buff remove — removals of buffs,

  • Buff apply — applications of buffs,

  • Buff damage — damage being inflicted by buffs,

  • Direct damage — damage being inflicted with skills directly,

  • State change — various; see the following section.

                                         17
State changes
State changes are a type of combat item both used to save metadata and record
many events that happen within combat. The type of the state change determines
what the contents mean. Table 2.2 lists all state changes as of version 20210103.
    Multiple state changes are only included once per interval if the represented
value changed, typically 0.3 seconds. This includes POSITION, VELOCITY, FACING,
HEALTHUPDATE, BREAKBARPERCENT. Including these more often would result in a
significant log size increase.
    Some of these state changes require further explanation, which follows.

Log start The LOGSTART state change includes timestamps that may be used
to determine when the log was recorded. The log includes the system time of
the computer used to record the log and the server time as tracked by the game
server. While recorded logs may include the time of generation in their filename,
the local timezone is used. If the user changes timezones, the time in the filename
would then be offset.

Reward The REWARD state change encodes reward chests being awarded to play-
ers. In the game, these are situated in the lower right corner of the UI and have
to be opened by the player to receive rewards. The ID and type of reward is also
encoded.
    It is worth noting that this includes chests that are awarded as a daily login
reward. These chests are not only given at login, but also awarded at daily reset,
which happens at 00:00 UTC. Not checking the reward ID or type can result in
false positives if rewards are used to determine if an encounter was successful.

Team change An agent may swap teams. This changes who they are hostile
towards and which agents may be affected by their skills. The TEAMCHANGE state
change includes the ID of the team an agent now belongs to.
    There are multiple encounters where this happens with a player. Keeping
track of this state change is, however, not required in order to properly keep
track of hostility as most combat items include a byte that specifies whether a
target is currently a foe.

Guild The GUILD state change only includes a GUID of the guild a player is
representing. The name of the guild and other information is not included; it
may, however, be retrieved from the official Guild Wars 2 API [13] by using this
GUID.

2.3      Limitations
As arcdps creates EVTC logs by using data that is present in the game client,
there are various limitations and inaccuracies in the recorded values. The game
client does not receive all data necessary to correctly estimate damage inflicted
by players, and there are other significant limitations we describe in this section.

                                        18
State change   Description                                EVTC Version
    ENTERCOMBAT    An agent entered combat.
     EXITCOMBAT    An agent exited combat.
       CHANGEUP    An agent is no longer downed1 .
     CHANGEDEAD    An agent has died.
     CHANGEDOWN    An agent has been downed.
          SPAWN    An agent has entered tracking range.
        DESPAWN    An agent is no longer tracked.
   HEALTHUPDATE    A health percentage for an agent.
       LOGSTART    First combat item of the log.
         LOGEND    Last combat item of the log.
       WEAPSWAP    An agent has swapped weapon sets.
MAXHEALTHUPDATE    An agent maximum health change.
    POINTOFVIEW    Encodes the recording player.
       LANGUAGE    Encodes the game language.
        GWBUILD    Encodes the version of the game.
        SHARDID    Encodes the ID of the server instance.
         REWARD    A reward chest has been awarded.
    BUFFINITIAL    A buff already present on an agent.
       POSITION    A position for an agent.                   >20180626
       VELOCITY    A velocity for an agent.                   >20180626
         FACING    A facing direction for an agent.           >20180718
     TEAMCHANGE    A team change for an agent.                >20181002
   ATTACKTARGET    Encodes an attack target relationship.     >20181002
     TARGETABLE    An attack target targetability change.     >20181002
          MAPID    Encodes the ID of the map.                 >20181008
       REPLINFO    Internal; should not appear in logs.
    STACKACTIVE    A buff stack has become active.            >20181214
     STACKRESET    A buff stack is reset.                     >20190108
          GUILD    Encodes the guild GUID of a player.        >20190329
       BUFFINFO    Encodes data for a buff skill.             >20191225
    BUFFFORMULA    Encodes a formula for a buff skill.        >20191225
      SKILLINFO    Encodes detailed skill info.               >20191225
    SKILLTIMING    Encodes animation timing for a skill.      >20191225
  BREAKBARSTATE    A state update of an agent defiance bar.   >20200506
BREAKBARPERCENT    A percentage for an agent defiance bar.    >20200506
          ERROR    Encodes an error message.                  >20200513
            TAG    Encodes a tag2 visible above a player.     >20200609
  BARRIERUPDATE    A barrier3 percentage of an agent.         >20201201

                   Table 2.2: A list of all state changes.

                                     19
2.3.1     Reporting range
The game client only knows about agents within a limited notification area. If
an agent goes out of range, it is no longer tracked and the client does not know
about events that impact them anymore. This limitation only applies to NPCs;
gadgets are visible at any distance. An agent moving out of the notification area
and back inside results in a second, separate agent being added to the log because
arcdps cannot detect this is the same agent as before. A DESPAWN state change is
issued as an agent leaves the notification area.

2.3.2     Missing damage
There are two sources of damage which are not detected by arcdps, and thus do
not appear in EVTC logs:
   1. attacks which inflict a percentage of maximum health, typically used by
      enemy NPCs;
   2. life siphoning, unless arcdps detects it indirectly.
    There are many player abilities utilizing life siphoning, and arcdps attempts
to detect such damage indirectly.
    Some supported siphons only correctly detect the attacker, but not the target.
An example of this would be Battle Scars, a stacking buff which siphons health
and consumes one stack of the buff every time an enemy is hit. In this case, arcdps
is unable to reliably detect the target as the server sends separate notifications
for damaging the enemy and a stack being removed. Due to network latency and
timing imperfections, it is not possible to reliably pair these notifications.
    Some life siphons are fully missing in EVTC logs, as of version 20210103 this
includes:
   • siphons granted by some Nourishment effects (food of ascended rarity),
   • siphons granted by weapon sigils,
   • siphons granted by many traits.
    Life siphons are typically not a significant portion of damage, and the impact
of this limitation is fairly minor.

2.3.3     Conditions
The game server does not notify the game client of damage inflicted by status
effects applied by other players, as this damage is not shown in the user interface
of the game. Status effects which cause damage over time in Guild Wars 2 are
standardized to a few types1 and are referred to as conditions.
    The addon solves this issue by simulating all condition damage by tracking
effect applications and calculating expected damage throughout time. This in-
troduces a potential desynchronization with reality in case the arcdps calculation
of condition damage is not correct.
  1
    Damaging conditions are Bleeding, Burning, Confusion, Poisoned, and Torment. The me-
chanics of these conditions slightly differ.

                                          20
The accuracy of conditions calculations may be negatively affected if the sim-
ulation thread is slowed down, which may happen with weaker computers. If
arcdps detects this issue, an ERROR state change is emitted to the EVTC log.

Missing Confusion
Confusion is a condition which only inflicts a small amount of damage over time,
but has an extra feature of damaging the enemy if the enemy casts a skill.
    In order to implement simulation for this condition, arcdps has to detect skill
activations. The game server, however, does not send notifications for skills which
feature no animation of the enemy model.
    This is most notable on two encounters logged by default:

  • Mursaat Overseer — all confusion damage is missing, even though the
    enemy casts a skill at a rate of once per 2.6 seconds;

  • Soulless Horror — approximately half of confusion damage is missing as
    many attacks of the enemy do not have an animation.

   This limitation causes significant issues as many players are unaware of it,
and confusion may form over 60 % of damage output of some game professions.

Level scaling
Player characters in Guild Wars 2 have a level indicating their strength. Most
content logged by arcdps is aimed at characters of level 80, the maximum level.
The game features a mechanism for scaling down player stats if they choose to
play content aimed at lower levels.
    The arcdps addon does not have correct calculations of stats for characters
scaled to lower levels, which makes condition damage calculations imprecise.
    This is notable for Dungeons — instances for groups of 5 players. Dungeon
levels range from level 30 to level 80.

2.3.4    Healing
The game client is unaware of healing performed by other players unless it directly
affects the player. It is also not notified about overhealing, which occurs when
a portion of a healing effect is effectively wasted because the target was at full
health.
    This is a very significant limitation, as healing is often analyzed in other
games. EVTC logs do not include healing statistics at all.

2.3.5    Buff extensions
The game features skills which extend durations of existing effects applied to
agents. EVTC logs include extensions, but do not contain the player causing the
extension as the game server does not send this data.

                                        21
2.4      Malformed logs
There are many cases of malformed EVTC logs with various causes. A common
cause is a lack of compatibility between arcdps and the game client after the
game is updated. There have also been many versions of arcdps which produced
invalid EVTC logs.
    Another common source of malformed logs is running two instances of the
game client at the same time. If both game instances participate in the same
fight, there is a big chance of EVTC data being corrupted as both instances of
arcdps try to write the same file at the same time. Notable issues in logs and
versions they affect include:

   • missing log start events (20181211),

   • missing log end events (throughout year 2018),

   • duplicate log end events (20200506),

   • cycles in minion-master hierarchy (throughout year 2017),

   • random big maximum health values (such as 263 ; throughout year 2018),

   • duplicate skill definitions (throughout year 2017, typically a non-existent
     skill with ID 0),

   • missing agent names (incompatible arcdps and Guild Wars 2 versions, race
     conditions1 ; fairly common),

   • incorrect condition damage calculations (many versions, also commonly oc-
     curs when outdated),

   • missing agents other than the main target (occurs if no names of agents are
     available),

   • wrong skill count (20210102);

   • . . . and many others.

    Any implementations reading EVTC logs need to be programmed defensively,
including ours.

   1
    Agent names are encrypted in the game files and the server sends a decryption key when
the agent appears on the map. This process may take a few milliseconds which leads to timing
issues.

                                            22
Part III

Design

   23
3. EVTC library
In this chapter, we detail the design choices we have made for the library for
consuming and analyzing EVTC log data.

3.1      Data representation
Exposing raw values from EVTC logs would result in a hard-to-use interface that
requires full knowledge of the way values are encoded, which is not straightforward
in the EVTC format.
    We instead introduce an abstraction in our API, providing .NET objects which
only contain relevant values. We refer to these objects as processed. This abstrac-
tion comes at the cost of one-to-one mapping to raw EVTC values.
    However, access to raw values is also desirable as it allows direct editing of
logs, and we make use of this mechanism to create logs for testing the library as
well. We further detail our approach to testing in section 3.5.
    For these reasons, the library facilitates a workflow with the following separate
steps:

  1. Parsing: read raw data and provide objects containing full raw data from
     encoded structs,

  2. Processing: process objects with raw data into processed objects

  3. Analysis: calculate statistics from processed objects.

3.2      Step 1: Parsing
The parsing step provides an abstraction over the layout of binary data in EVTC
logs.

3.2.1     Revision compatibility
We provide support for the existing revisions 0 and 1 of the EVTC format, and
provide no guarantees for future revisions. We instead plan to find a solution
depending on how significant the changes to the binary format are if a new revision
is specified.
    The first two revisions (0 and 1) only differ in the layout of the event structs,
thus we define just one class to represent it – the ParsedCombatItem, as the data
of revision 0 is a subset of revision 1 data.

3.2.2     Data container
All raw data from a log is stored in a ParsedLog object and we provide a
EVTCParser class for producing these objects from log files. This effectively sep-
arates the parsing and processing steps, and the ParsedLog is the only object
shared between these two layers.

                                         24
This approach has turned out to provide an easy-to-consume API, but at
the cost of performance, as many objects with raw data are created only to be
provided to the processing layer and promptly discarded once processed. This
results in a strain on the garbage collector of the .NET runtime when processing
many logs. As a result, we have identified a need for another API that directly
passes raw read-only data to the processing layer and reuses memory. This may
be implemented in the future as part of further optimization work.

3.2.3      Editing
The raw data layer also provides easy access to log data manipulation. We provide
an EVTCWriter that accepts a ParsedLog and outputs a revision 1 EVTC log file.
This API is generic enough to allow creating and editing any EVTC log while
abstracting away the binary layout of structs of the EVTC format.

3.3       Step 2: Processing
The processing step converts raw data from the parsing step into processed ob-
jects which only contain relevant values. We provide a LogProcessor capable of
performing this step and producing immutable Log objects — the sole output of
this step.

3.3.1      Relational data
EVTC events often contain relational data, such as agents causing the event
or being affected by it and, in many cases, master-minion relations of agents.
The processing layer constructs objects with these relations resolved instead of
requiring users of the library to manually handle mapping IDs of agents and skills.

3.3.2      Encounter data definitions
We also provide custom definitions for many distinct encounters that are logged
by arcdps by default. To provide support for varying encounters, we define an
IEncounterData interface which provides the following encounter-specific lazy
definitions:

   • result identification (i.e. success or failure) and encounter duration calcu-
     lation1 , adhering to an IResultDeterminer interface;

   • mode identification (e.g. normal mode, challenge mode2 ), adhering to an
     IModeDeterminer interface;

   • health of the main enemy at the time of failure used to show progress of
     logged failures, adhering to an IHealthDeterminer interface;
   1
     Encounter duration is significantly tied to results. EVTC logs often continue for multiple
seconds after the encounter is considered successfully finished because players do not exit combat
instantly.
   2
     Many encounters within Guild Wars 2 have versions with higher difficulty. These are known
as Challenge Modes, often abbreviated as CM.

                                               25
• main enemy targets;
  • post-processing steps further described in section 3.3.3.
    We choose to provide lazy definitions instead of evaluating the values in the
processing phase. The user of the library may not need all values, and only
calculating what is needed is a significant performance gain. The definitions are
instead evaluated during the analysis step.
    We are, however, required to perform encounter identification in this phase
because of a limitation of EVTC logs — NPC splitting detailed in section 2.3.1.
We need to identify the encounter at this stage to safely merge these agents back
into one. This merging significantly simplifies logic related to such agents.
    To identify encounters, we define an IEncounterIdentifier interface, and
provide an implementation with support for all encounters logged by default.
The IEncounterIdentifier used by the LogProcessor may be replaced by con-
sumers of the library to add support for extra encounters or to override the default
definitions.
    These are all definitions that commonly need to be redefined for encounters
to account for their unique design. We also offer a default implementation for
unknown encounters, but that is very likely to be incorrect unless the encounter
follows the most basic formula. This is one of the reasons why we offer the
option to redefine these definitions to consumers of the library by replacing the
IEncounterIdentifier.
    We also provide reusable implementations of the determiner interfaces, such
as AgentDeadResultDeterminer, to prevent code duplication for encounters with
similar mechanics.
    We choose not to make use of REWARD events to detect success unless they are
awarded every time the encounter is finished. Many encounters only provide a
reward the first time they are finished in a period of time, usually a week. The
relative scarcity of reward-less logs increases the likelyhood of rare bugs in success
detection for logs without rewards.

3.3.3     Post-processing steps
IEncounterData mostly provides definitions that are only evaluated as needed
during the analysis step. The only exception are post-processing steps adhering
to the IPostProcessingStep interface. The LogProcessor maintains its state
within a mutable LogProcessorContext object, which is provided for modifica-
tion to the Process method defined by the interface.
    The only post-processing step defined by the library is MergeSingletonNPC,
used for merging of NPCs in encounters where they are likely to get split into
multiple agents as described in section 3.3.2. This step should only be used on
species of NPC guaranteed to only appear once in the encounter.
    This mechanism is the last point at which the processed log may be modified,
as the final resulting Log object is fully immutable.

3.4      Step 3: Analysis
The last step encompasses all actions done on the immutable Log object.

                                         26
We provide a LogAnalyzer class with convenience methods for calculating
common statistics and cache these results. Most provided statistics use defini-
tions from IEncounterData as detailed in section 3.3.2. New statistics may be
introduced to the LogAnalyzer over time if they are general enough.

3.5      Testing
Correctness of the results is expected from consumers of the library. Automatic
testing allows us to easily maintain a good level of output quality.
    There are many approaches to testing. Unit tests may be used to verify
individual parts of the library. We make use of this mechanism where applicable.
It is, however, impractical to test correctness this way for behavior as complex
as the combat described in EVTC logs.
    As we have detailed in section 2.4, real log files may contain various abnormal-
ities, ranging from minor issues to unsalvageable situations. Our library should
properly handle these cases.
    We base the main part of our test suite on a collection of real logs generated
by the arcdps addon. In order to verify correct functionality of our library, we
maintain a set of logs with verified results and compare the results from our
library.
    We also build a testing project which compares our results with Elite Insights,
the only other implementation with reliable results. We expect these tests to find
issues in both our implementation and in Elite Insights.
    We may also make use of log editing functionality detailed in section 3.2.3.
Removal of REWARD state changes can be used to remove rewards which are only
provided when successfully finishing an encounter for the first time in a week.
This gives us a bigger dataset for repeated encounters, which also need reliable
success detection without relying on REWARD state changes. This will mainly result
in improvements to Elite Insights as it makes use of these events.

                                        27
4. Log Manager
In this chapter, we describe the choices we have made during development of the
arcdps Log Manager program.
    We find it important to note that the development of our project has been
ongoing for 3 years and some choices were made according to options available
as of 2018. Some design choices were changed throughout development according
to new options and user feedback from an early group of alpha users.

4.1      Choosing a GUI framework
The log manager should be a desktop application with a graphical user interface.
To build it, we first need to choose a .NET GUI library that best fulfills our
requirements:
   • Cross-platform compatibility (Windows, Linux, optionally others). This re-
     quirement is mandatory as it is required to fulfill the (C2) goal.

   • Availability of controls. The library needs to provide sufficient user interface
     elements — controls. As we aim to emulate a file manager, a feature-rich
     grid control for building tables is very valuable.

   • High performance. We plan to support very high amounts of logs to fulfill
     the (C4) goal. For this reason, we need a sufficiently performant library.
     This includes the need for the library to provide virtualized rendering1 for
     the list of logs. Additionally, a low memory footprint is also a significant
     benefit.

   • Production readiness. A production-ready library has benefits in compar-
     ison to libraries in early development. API stability, a smaller likelihood
     of bugs in the library, and better documentation all contribute to easier
     development. The likelihood of the library being abandoned is also lower.

   • Ease-of-use. The library should preferably be easy to use, and good tooling
     should be available.

   • Native look. The interface should be easy to understand for users. The use
     of native controls contributes to this, but is not critical.
    We have evaluated multiple GUI framework and library options. The results
of our evaluation follow:
   • Windows Forms is a UI framework included as part of the .NET Frame-
     work and .NET Core 3 and newer. It does not provide cross-platform
     compatibility, as it is only available on Windows in its .NET Framework
     and .NET Core 3 implementations. The Mono Project version of the .NET
     Framework provides a rudimentary cross-platform Windows Forms imple-
     mentation. This implementation is largely unmaintained and appears to be
  1
    Virtualized rendering is a technique that prevents drawing data that does not appear on
the screen, typically in the context of big tables.

                                            28
discontinued after the Mono Project acquisition by Microsoft. The Mono
  implementation of Windows Forms also has not been ported to .NET Core,
  which reduces options to update in the future. For these reasons, we reject
  it as an option.

• Windows Presentation Foundation (WPF) is a UI framework that is in-
  cluded as a part of the .NET Framework and .NET Core 3 and newer.
  Similarly to Windows Forms, it is also only available on Windows in both
  its .NET Framework and .NET Core 3 versions. We reject this option
  because of this limitation.

• Avalonia [14] is an open-source cross-platform .NET UI framework. Aval-
  onia uses custom rendering for its interface, which results in a non-native
  look. We have decided that this limitation is acceptable, and attempted
  to build a prototype of the user interface. We have ran into multiple bugs
  in basic functionality, including broken scrolling in its DataGrid control,
  which would be a critical part of the application. We have also observed
  poor performance. For these reasons, we have identified the project to not
  be production-ready at this point and decided not to use it.

• Electron.NET [15] is a cross-platform wrapper that embeds a local web
  server written in .NET and runs a Chromium-based browser to access this
  web server. This would allow us to build the backend of the application us-
  ing web frameworks, such as ASP.NET. The frontend would be implemented
  using HTML, CSS, and Javascript-based technologies. This introduces ad-
  ditional complexity, and is known for introducing extra performance over-
  head, especially for memory use.

• Xamarin.Forms [16] is a cross-platform UI Framework with a native look
  on all its platforms. At the time of making this choice, no Linux version
  was available.

• GtkSharp [17] is a wrapper for the Gtk library [18] for building user in-
  terfaces. This library is low-level in comparison to the previously listed
  frameworks, which would increase development time. The available doc-
  umentation is very limited, which includes documentation for using this
  library on Windows. Usage of this library would also result in a native look
  on Linux, but not on Windows.

• Xwt [19] is a cross-platform framework developed by the Mono Project
  with a native look on all supported platforms. The framework only offers a
  small amount of user interface controls, and misses any kind of grid or table
  that may be used for the main part interface. Furthermore, the framework
  appears to be superseded by Xamarin.Forms, and has not received new
  features in a long time.

• Eto.Forms [20] is a cross-platform framework with a native look on all plat-
  forms with a single codebase for the UI. The framework supports using
  Windows Forms or WPF as the platform for running on Windows, GTK as
  the platform for running on Linux, and MonoMac as the platform for ma-
  cOS. Eto.Forms provides a big selection of user interface controls, including

                                    29
a feature-rich GridView. There is a testing project showcasing the use of all
      controls, and extra documentation available. The API is generally easy to
      use and resembles Windows Forms, with extra capabilities similar to WPF.
      The performance is mostly dictated by the underlying platform, which is
      sufficient for our use case with the options available.
      To test the quality of Eto.Forms, we have decided to implement a basic
      prototype, similarly as we did with Avalonia. We found the implementation
      very reliable and encountered no issues during this stage. The development
      of the framework is ongoing, which is promising in case we encounter bugs
      later in the development. The main weakness appears to be the relative
      obscurity of this library in comparison to other options.

  After evaluating all of the listed options, we have chosen Eto.Forms as our
GUI framework as it fulfills our requirements the best.

4.2      User interface
The main function of the log manager is providing an efficient way of browsing
EVTC logs. A standard file explorer does not have insight into the contents of
the log file, and is only capable of showing the file name, date of creation and file
size, which is not sufficient for efficiently finding logs.
    We have identified the following information as crucial for efficiently searching
logs:

  • the name of the encounter,

  • the date and time of the encounter,

  • the result of the encounter, i.e. success or failure,

  • the mode of the encounter, e.g. normal or challenge,

  • the duration of the encounter,

  • the players participating in the encounter.

    We draw inspiration from traditional file explorers, such as the Windows File
Explorer, Dolphin [21], GNOME Files [22], and similar. As our main goal is
efficiency, we are interested in views named detail view or list view in these file
explorers. These views list items in a table, with a row for each item and columns
showing properties such as names, dates, and similar. We base the central com-
ponent of our manager — the log list — off this design.
    However, showing a list of player names in such a table is infeasible as logs
commonly include 5–10 players for PvE logs and up to 50 players for WvW logs.
We instead show the composition of players in the table by using known icons
representing the game professions of player characters. The full list of players,
including character and account names, is only shown for a selected log. For this
purpose, we introduce a pane with details about a selected log. This pane will
not only show details, but also allow manipulation of selected logs.

                                         30
You can also read