Norton 360 for Gamers Game Optimization Testing
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Norton 360 for Gamers Game Optimization Testing Windows 10 June 2021 Document: Norton 360 for Gamers Game Optimization Testing (Edition 1) Authors: J. Han, D. Wren Company: PassMark Software Date: 22 June 2021 Edition: 1 File: Norton_Game_Optimization_Testing_2021_Ed1.docx
Norton 360 for Gamers Game Optimization Testing PassMark Software
Table of Contents
REVISION HISTORY............................................................................................................................................ 3
EXECUTIVE SUMMARY ...................................................................................................................................... 4
OVERALL SCORE ................................................................................................................................................ 5
PRODUCTS LIST ................................................................................................................................................. 6
TEST RESULTS.................................................................................................................................................... 7
PHASE 1 ......................................................................................................................................................... 7
PHASE 2 ....................................................................................................................................................... 15
CONCLUSION .................................................................................................................................................. 17
SCENARIOS WHERE THERE WAS A BENEFIT .............................................................................................................. 17
SCENARIOS WHERE THERE WAS NO BENEFIT ............................................................................................................ 17
DISCLAIMER AND DISCLOSURE ....................................................................................................................... 18
DISCLAIMER OF LIABILITY................................................................................................................................... 18
DISCLOSURE ................................................................................................................................................... 18
TRADEMARKS ................................................................................................................................................. 18
CONTACT DETAILS........................................................................................................................................... 18
APPENDIX 1 – TEST ENVIRONMENT ................................................................................................................ 19
TESTING MACHINE .......................................................................................................................................... 19
TESTING GAMES ............................................................................................................................................. 19
APPENDIX 2 –METHODOLOGY DESCRIPTION .................................................................................................. 20
WINDOWS 10 IMAGE CREATION ......................................................................................................................... 20
GAME BENCHMARK TEST .................................................................................................................................. 20
TEST PHASE 1................................................................................................................................................. 20
TEST PHASE 2................................................................................................................................................. 21
Edition 1 Page 2 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Revision History
Rev Revision History Date
Edition 1 Initial version of report 22 June 2021
Edition 1 Page 3 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Executive Summary
PassMark Software conducted objective performance testing of the Game Optimizer (GO) feature in the Norton
360 for Gamers product.
The performance benchmark test consisted of two phases. The aim of the first phase was to test the effect of GO
on various games, both on a clean install of Windows 10 (Phase 1.1) and also with background activity (Phase 1.2).
The second phase (Phase 2) is to compare the Norton 360 for Gamers product with competing Anti-virus products,
which also advertised similar game optimizing features.
To test the game optimization performance, we selected five common PC games and used the games built-in
benchmarks to measure the minimum, maximum and average FPS (frames per second) metrics. We ran some
background tasks in parallel with the games to test the Norton GO game optimizing performance and we used a
synthetic CPU load tool to produce a constant CPU consumption when comparing the Norton 360 for Gamers with
the competitors’ products.
We found that Norton GO improves frame rate performance in games in some scenarios. It is of particular benefit
when there are high CPU usage activities running in the background during the game play and the CPU is creating
a bottleneck. In comparison to similar anti-virus products, Norton 360 for Gamers performed the best in the
product comparison. Further details are in the report below.
Edition 1 Page 4 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Overall Score
PassMark Software assigned a score to each product we tested in the Phase 2 test, depending on its ranking of the
average FPS metric compared to other products in the category.
In the following table the highest possible score attainable is 30; in a hypothetical situation where a product has
attained first place in all 5 games. The table below shows the products ranked by their overall scores:1
Product Name Overall Score
Norton 360 for Gamers 30
Webroot SecureAnywhere AntiVirus for PC Gamers 20
ESET NOD32 Antivirus 17
McAfee Gamer Security 16
Kaspersky Total Security 11
Bitdefender Antivirus Plus 9
1
The Razer Cortex product was not included in the comparison as it did not contain any anti-virus functionality.
Edition 1 Page 5 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Products List
The tested game optimization product names and versions are as follows:
Product Name Version
Norton 360 for Gamers 22.21.5.44
Razer Cortex 9.14.15.1361
McAfee Gamer Security 1.7.128
ESET NOD32 Antivirus 14.1.20.0
Bitdefender Antivirus Plus 25.0.21.78
Kaspersky Total Security 21.3.10.391 (b)
Webroot SecureAnywhere AntiVirus for PC Gamers 9.0.30.75
Edition 1 Page 6 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Test Results
We performed two phases of testing. The results obtained from each phase test have been plotted on charts for
ease of comparison.
Phase 1
In the first stage of the Phase 1 test, we ran the frame rate benchmarks on a clean install of Windows 10 without
Norton GO installed and also with the Microsoft Defender turned off. Then the test was repeated with the Norton
GO active to compare the two results.
No Background Task
46.82
Ghost Recon Wildlands
47.04
59.90
Sleeping Dogs
59.90
59.00
Forza Horizon 4
59.00
35.00
Shadow of the Tomb Raider
35.00
42.00
Far Cry 5
42.00
0 10 20 30 40 50 60 70
Average FPS
Norton GO not installed Norton GO installed
Note that the above test was ran with NVIDIA GeForce GTX 960 graphic card, and subsequently we replaced the
card with NVIDIA GeForce RTX 2060 and used the new graphic card for the remaining of the tests. This was done
to remove the GPU as a significant bottleneck.
Edition 1 Page 7 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
In the second stage of the Phase 1 test, we ran one or more third-party applications in parallel with the games.
The types of background tasks were:
• Downloading games on Steam
• Chrome loading Web pages
• Video encoding with Handbrake
• Game recording with Game DVR
• Chrome Cleanup computer
• CPU stress with GBCPUStressTest
These background activities produced different levels of CPU loads on the system while running games
benchmarks.
We ran the background tasks benchmarks with no Norton GO product installed (Microsoft Defender was turned
off). Then the test was repeated with the Norton 360 for Gamers installed to test the effect of game enhancement
feature of the Norton GO. We also ran the benchmarks with no background tasks, Norton GO not installed, and
the Microsoft Defender turned off to produce a baseline. The average percentage increases in the average FPS
metric achieved by the Norton GO are shown in the table:
Shadow of the Forza Ghost Recon
Background Tasks Far Cry 5 Sleeping Dogs
Tomb Raider Horizon 4 Wildlands
Downloading games on Steam 30.57% 13.77% 3.38% 5.15% 11.76%
Chrome loading Web pages 0.35% 2.22% 0.86% 0.86% -0.21%
Video encoding with Handbrake 40.69% 35.68% 13.75% 30.43% 24.61%
Game recording with Game DVR 0.66% 0.69% 0.00% -0.54% 0.56%
Chrome Cleanup computer 5.88% -3.13% 2.31% 3.10% 1.92%
CPU stress with GBCPUStressTest 43.63% 19.09% 6.43% 10.75% 47.93%
The Average Percentage Increases in the Average FPS by Norton GO
Edition 1 Page 8 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
The details of the average FPS measured from each type of background task test are shown in the chars below:
Background Task 1 - Downloading games on Steam
98.37
Ghost Recon Wildlands 79.54
88.89
140.97
Sleeping Dogs 125.63
132.10
125.00
Forza Horizon 4 118.33
122.33
100.67
Shadow of the Tomb Raider 82.33
93.67
103.67
Far Cry 5 76.33
99.67
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 9 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 2 - Chrome loading Web pages
98.37
Ghost Recon Wildlands 91.01
90.82
140.97
Sleeping Dogs 131.30
132.43
125.00
Forza Horizon 4 116.00
117.00
100.67
Shadow of the Tomb Raider 90.00
92.00
103.67
Far Cry 5 95.33
95.67
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 10 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 3 - Video encoding with Handbrake
98.37
Ghost Recon Wildlands 75.61
94.22
140.97
Sleeping Dogs 99.90
130.30
125.00
Forza Horizon 4 106.67
121.33
100.67
Shadow of the Tomb Raider 71.00
96.33
103.67
Far Cry 5 68.00
95.67
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 11 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 4 - Game recording with Game DVR
98.37
Ghost Recon Wildlands 94.47
95.00
140.97
Sleeping Dogs 135.27
134.53
125.00
Forza Horizon 4 120.00
120.00
100.67
Shadow of the Tomb Raider 96.33
97.00
103.67
Far Cry 5 100.67
101.33
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 12 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 5 - Chrome Cleanup computer
98.37
Ghost Recon Wildlands 91.89
93.66
140.97
Sleeping Dogs 127.90
131.87
125.00
Forza Horizon 4 115.33
118.00
100.67
Shadow of the Tomb Raider 96.00
93.00
103.67
Far Cry 5 90.67
96.00
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 13 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 6 - CPU stress with GBCPUStressTest
98.37
Ghost Recon Wildlands 63.74
94.29
140.97
Sleeping Dogs 123.67
136.97
125.00
Forza Horizon 4 114.00
121.33
100.67
Shadow of the Tomb Raider 80.33
95.67
103.67
Far Cry 5 68.00
97.67
0 20 40 60 80 100 120 140 160
Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 14 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Phase 2
We performed the second phase of the test with different GO products installed and compared the results to each
other. We manually produced a fixed amount of CPU load on the system while launching the game’s benchmark
mode. A third-party CPU stress tool GBCPUStressTest was used to ensure the CPU load was constant for all
competitors tests.
In the following charts, we have highlighted the results we obtained for Norton 360 for Gamers in orange color
and the average has been highlighted in blue. Products with higher average FPS values are considered better
performing products.
Game 1 - Far Cry 5
Bitdefender Antivirus Plus 66.00
Kaspersky Total Security 68.67
ESET NOD32 Antivirus 69.67
Webroot SecureAnywhere AntiVirus for PC Gamers 71.33
Average 76.22
McAfee Gamer Security 84.00
Norton 360 for Gamers 97.67
0 20 40 60 80 100 120
Average FPS
Game 2 – Shadow of the Tomb Raider
Bitdefender Antivirus Plus 81.00
McAfee Gamer Security 82.67
Webroot SecureAnywhere AntiVirus for PC Gamers 84.33
Kaspersky Total Security 84.67
ESET NOD32 Antivirus 84.67
Average 85.83
Norton 360 for Gamers 97.67
0 20 40 60 80 100 120
Average FPS
Edition 1 Page 15 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Game 3 – Forza Horizon 4
McAfee Gamer Security 112.67
Kaspersky Total Security 114.67
ESET NOD32 Antivirus 114.67
Bitdefender Antivirus Plus 115.00
Webroot SecureAnywhere AntiVirus for PC Gamers 115.67
Average 115.83
Norton 360 for Gamers 122.33
0 20 40 60 80 100 120 140
Average FPS
Game 4 – Sleeping Dogs
Kaspersky Total Security 117.83
Bitdefender Antivirus Plus 117.90
Webroot SecureAnywhere AntiVirus for PC Gamers 119.33
ESET NOD32 Antivirus 119.73
Average 122.44
McAfee Gamer Security 123.87
Norton 360 for Gamers 135.97
0 20 40 60 80 100 120 140 160
Average FPS
Game 5 – Tom Clancy's Ghost Recon Wildlands
Bitdefender Antivirus Plus 58.15
Kaspersky Total Security 59.18
McAfee Gamer Security 60.58
ESET NOD32 Antivirus 61.33
Webroot SecureAnywhere AntiVirus for PC Gamers 61.59
Average 66.02
Norton 360 for Gamers 95.30
0 20 40 60 80 100 120
Average FPS
Edition 1 Page 16 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Conclusion
We found that Norton GO could help improve frame rate performance in games in some scenarios.
Scenarios where there was a benefit
GO is of particular benefit when there are high CPU usage activities running in the background during the game
play and the CPU is creating a bottleneck. It is important to note that this benefit was limited to restoring lost
performance due to background CPU load and there was no scenario where game performance exceeded the
performance obtained with a clean install of Windows.
Scenarios where there was no benefit
GO’s method of functioning only acts on the CPU’s process scheduling (known as process affinity and process
scheduling priorities). The theory of its functioning is that the full resources of the system can be given to the
game, at the expense of any background tasks. So, in the following cases no benefit or limited benefit can be
expected.
• CPUs with only one or two cores. You can’t reschedule background tasks to another core if there is no
other core available.
• CPUs with large numbers of cores. e.g. 8+ cores. In this case many CPU cores are likely already idle. So
rescheduling achieves nothing.
• Gaming PCs with a relatively slow video card in comparison to the CPU. In this case the video card is the
bottle neck.
• Gamers running very high resolution monitors with games at maximum quality settings. In this case the
video card is more often the bottle neck.
• PCs with no significant background activity. e.g. a clean install of Windows.
• PCs with significant background activity, but that background activity is mostly placing load on the disk,
internet or video card. If there is no CPU load then the CPU load can’t be optimized.
• Games that are not identified as games by GO. For example rare games, or games that don’t run full
screen. In this case GO may not activate. However we did not see this in our testing. All games tested
were identified as games.
Edition 1 Page 17 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Disclaimer and Disclosure
This report only covers the versions that are listed in the “Products List” section of this report.
Disclaimer of Liability
While every effort has been made to ensure that the information presented in this report is accurate, PassMark
Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable
in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from the
availability of, use of, access of, or inability to use this information.
Disclosure
NortonLifeLock Inc. funded the production of this report. The products list, performance metrics, and the games
list included in the report were selected by NortonLifeLock.
Trademarks
All trademarks are the property of their respective owners.
Contact Details
PassMark Software Pty Ltd
Level 5, 63 Foveaux St
Surry Hills, 2010
Sydney, Australia
Phone + 61 (2) 9690 0444
Fax + 61 (2) 9690 0445
Web www.passmark.com
Edition 1 Page 18 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Appendix 1 – Test Environment
Testing Machine
The test performed on a desktop computer with the following hardware specifications:
Motherboard: ASRock Fatal1ty Z170 Gaming K6
CPU: Intel Core i7-6700K @ 4.00GHz
RAM: G.Skill DDR4-2133MHz 2x8GB
Video Card: ASUS Strix GeForce GTX 960 4GB
Video Card: Gigabyte GeForce RTX 2060 OC 6GB
Main Drive (OS): Samsung 950 Pro M.2 NVMe 256GB
nd
2 Drive (Steam Library): Samsung 850 Pro SATA III 512GB
Monitor1: Dell UltraSharp U2412M (1920 x 1200 at 59 Hz)
Monitor2: Dell UltraSharp U2412M (1920 x 1200 at 59 Hz)
Internet: 1Gbps Internet Connection
OS: Windows 10 Pro Version 2004
We used dual monitors with extended display mode, one monitor for playing the games in full-screen mode and
the other one for executing and monitoring the background tasks.
NVIDIA GeForce GTX 960 graphic card was used for the first stage of Phase 1 test and NVIDIA GeForce RTX 2060
was used for the rest of the tests. NVIDIA graphics card Game Ready Driver (GRD) version 466.27 WHQL was
installed.
Testing Games
• Tom Clancy's Ghost Recon Wildlands
• Far Cry 5
• Shadow of the Tomb Raider
• Sleeping Dogs
• Forza Horizon 4
These games were selected as they are relatively modern and also have a built-in benchmark mode that returns
consistent results. The games were downloaded and installed using the Steam platform.
Edition 1 Page 19 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
Appendix 2 –Methodology Description
Windows 10 Image Creation
A bootable version of Macrium Reflect (Backup software) was used to create a “clean” baseline image prior to
testing. The aim was to create a baseline image with the smallest possible footprint and reduce the possibility of
variation caused by external operating system factors.
The baseline image was restored prior to testing of each different product. This process ensures that we install
and test all products on the same, “clean” machine.
The steps taken to create the base Windows 10 image were as follows:
1. Installation and activation of Windows 10 Pro Edition.
2. Update Windows.
3. Install and configure third-party applications and games used for testing.
4. Disable Microsoft Defender.
5. Disable Windows Automatic Updates.
6. Create a baseline image of the boot drive using Macrium Reflect.
Game Benchmark Test
The average FPS was measured by running the games’ built-in benchmark test modes. The game’s video and
graphic settings were configured as default (game application recommended settings) after entering each game
for the first time and the settings were kept the same for all tests of all products.
Each test was conducted three times and the final result was calculated as an average of the three samples.
Test Phase 1
The aim of the phase 1 benchmark is to test the effect of Norton GO, and this test consists of two parts. The first
part of the phase 1 test was conducted with no background tasks launched while playing games’ benchmark
modes. The second part of the phase 1 test was run with one or more third-party applications running in the
background with the game play.
Background Task 1 - Downloading games on Steam
This test was performed by enabling the “Allow downloads during gameplay” option from the Steam settings and
starting a game download and install before launching the game benchmark mode.
Background Task 2 - Chrome loading Web pages
This benchmark was conducted by playing two YouTube videos and loading ten other websites on Google Chrome
during the game play. The URLs are as follows:
URL
https://youtu.be/f8w7hATXQCg
https://youtu.be/ctObBrR0oN4
https://www.tradingview.com/chart/
Edition 1 Page 20 of 21
22 June 2021Norton 360 for Gamers Game Optimization Testing PassMark Software
https://au.finance.yahoo.com/
https://www.google.com/finance/
https://coinmarketcap.com/
https://www.binance.com/en/markets
https://www.news.com.au/
http://www.bom.gov.au/
https://news.google.com/topstories?hl=en-AU&gl=AU&ceid=AU:en
https://www.reddit.com/
https://www.passmark.com/
Background Task 3 - Video encoding with Handbrake
Command line version of HandBrake (Version 1.3.3) tool was used to encode an MP4 format sample video file to
MKV file format in the background when playing the game benchmarks.
Background Task 4 - Game recording with Game DVR
Before starting the game benchmark mode, the game recording was launched using the Windows 10 built-in tool
Game DVR.
Background Task 5 - Chrome Cleanup computer
This test was performed using the Google Chrome’s Cleanup tool. Clean up computer process was launched from
the Chrome Settings before starting the game benchmark.
Background Task 6 - CPU stress with GBCPUStressTest
A third-party CPU stress test tool GBCPUStressTest was used for this benchmark. The tool was configured to
produce 50% of CPU load to the system.
Test Phase 2
The aim of the phase 2 test is to compare the Norton 360 for Gamers with its competitors’ products.
GBCPUStressTest tool was used to produce the same amount of CPU load to all test runs of all products.
The products’ “Gaming Mode” was turned on manually if it was disabled by default.
Edition 1 Page 21 of 21
22 June 2021You can also read
