EVALUATING BLAZOR WEBASSEMBLY FOR THE PROGRESSIVE WEB APPLICATION FRONT-END - A COMPARATIVE STUDY USING REACTJS AS A BASELINE
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Evaluating Blazor WebAssembly for the Progressive Web Application Front-End A Comparative Study Using ReactJS as a Baseline Main Subject area: Computer Engineering Authors: Vida Rashidi, William Segelström Supervisor: Denis Golubovic JÖNKÖPING 2021 August
This final thesis has been carried out at the School of Engineering at Jönköping University within Computer Engineering. The authors are responsible for the presented opinions, conclusions, and results. Examiner: Johannes Schmidt Supervisor: Denis Golubovic Scope: 15 hp Date: 2021-08-19 i
Abstract This study is conducted to evaluate the Blazor WebAssembly framework for the Progressive Web Application (PWA) methodology. A comparative study is conducted with a ReactJS PWA as a baseline. The two frameworks are evaluated in their front-end performance and documentation of PWA- focused subjects. Front-end performance is measured between two experimental applications that test the loading times, heap memory usage and loading consistency during layout generation. It is found that a Blazor WebAssembly PWA takes on average a range of 0.34, 0.18, and 0.06 seconds less time to generate a layout than a ReactJS PWA. The Blazor WebAssembly PWA was less consistent in its loading times when handling many elements. Documentation that covers Progressive Web Application terminology was found to be different between the frameworks. The Blazor WebAssembly documentation covers more topics and provides first-hand knowledge while the ReactJS documentation covers fewer topics and relies on external sources to provide the necessary explanations. These findings indicate that Blazor WebAssembly is a faster framework when updating large amounts of elements in comparison to ReactJS. However, the ReactJS PWA was found to be overall more consistent in its loading times. Documentation varied between the two frameworks. Documentation in Blazor WebAssembly covers more Progressive Web Application subjects and is more in-depth than ReactJS. This study only evaluates applications developed in ReactJS and Blazor WebAssembly. Both are tested on Google Chrome in a desktop environment. Keywords: Blazor WebAssembly, .NET, WebAssembly, ReactJS, JavaScript, Progressive Web Application, Virtual DOM ii
Table of content Abstract ii 1 Introduction 1 1.1 Background 1 1.2 Problem statement 2 1.2.1 PWA and Web Frameworks 3 1.2.2 Layout Generation in a PWA 3 1.2.3 Why ReactJS Is Selected For Comparison 4 1.3 Purpose and research questions 4 1.3.1 Presentation of Research Questions 5 1.4 Scope and limitations 5 1.5 Disposition 6 2 Method and Implementation 7 2.1 RQ 1 - Literature Study 7 2.2 RQ 2 - Experiment Design and Execution 8 2.2.1 Implementation details 12 2.2.2 Specifications 16 2.2.3 Measuring Performance in Chrome DevTools 16 2.3 Method Motivation 17 2.4 Data collection 17 2.4.1 Gathering Performance Profiles for the Experiment 17 2.4.2 How the Experiment Is Automated 18 2.4.3 Collecting Information for the Literature Study 19 2.5 Data analysis 20 2.5.1 Experiment - Comparing Means of Independent Samples 20 2.5.2 Literature Study 22 2.6 Validity and reliability 22 2.6.1 Validity 22 2.6.2 Reliability 22 2.7 Considerations 23 2.7.1 Selection of Sources 24 iii
2.7.2 Implementation Fairness 24 3 Theoretical framework 27 3.1 Research Questions Related to the State of the Art 27 3.2 Technical Components of the Study 29 3.2.1 Progressive Web Application 29 3.2.2 WebAssembly 31 3.2.3 Virtual DOM 31 3.2.4 Blazor WebAssembly 32 3.2.4.1 C# Timer class 34 3.2.5 ReactJS 35 3.2.6 V8 JavaScript Engine 35 4 Results 38 4.1 PWA support for each framework 38 4.1.1 Blazor WebAssembly 39 4.1.2 ReactJS 41 4.2 Experiment Sessions 42 4.2.1 Session 1 - 720 pairs 42 4.2.2 Session 2 - 360 pairs 43 4.2.3 Session 3 - 180 pairs 44 4.3 Data Analysis 45 4.3.1 RQ 1 - Literature analysis 45 4.3.1.1 Analysis of results for Blazor Wasm 45 4.3.1.2 Analysis of results for ReactJS 45 4.3.2 RQ 2 - Experiment analysis 46 4.3.2.1 RQ 2.1 - Time difference 46 4.3.2.2 RQ 2.2 - Tree sizes 47 4.3.2.3 RQ 2.3 - Consistency in Loading Times 49 5 Discussion 51 5.1 Results discussion 51 5.1.1 PWA Support in the Blazor Wasm documentation 51 5.1.2 Virtual DOM differentiation 52 iv
5.2 Method discussion 54 5.2.1 When Analysing Memory From the Experiment 55 6 Conclusions and further research 57 6.1 Conclusions 57 6.1.1 Blazor WebAssembly as a PWA framework 57 6.1.2 Blazor WebAssembly and VDOM Generation 57 6.1.3 Practical implications 58 6.1.4 Scientific implications 58 6.2 Further research 58 7 References 59 8 Appendixes 64 Appendix 1 GitHub Repository of Source code for experiment PWAs and Python scripts used to collect and analyse retrieved data 64 v
1 Introduction In this section, the reader is introduced to the elements that are included in this study as well as the problem statement. This section also includes background information required for understanding the purpose of this study. Cross-platform development is the development of software that is not dependent on a specific platform. There are multiple methodologies for developing cross-platform software. One methodology proposed for cross-platform development is Progressive Web Applications (PWA). PWAs are web applications, which means that any web framework can be used to develop a PWA. The large selection of frameworks is making it difficult to select the most appropriate one for PWA development. This is because no established web framework is presented as a PWA-specific one. For this study, selected frameworks are scrutinized in the aspects of support for the PWA methodology and their front-end performance. The scrutiny is done by a comparative study between the two web frameworks. The two selected frameworks for this study are Blazor WebAssembly (Blazor Wasm) and ReactJS. The goal of this study is to create an understanding of how the selected frameworks support the PWA methodology and how well their front-end performs in a PWA runtime environment. 1.1 Background PWAs are web applications that can be installed locally. PWAs are installed by visiting a website that fulfils the PWA requirements and via the browser interface downloads the application. Figure 1 shows how the installation prompt appears to the user. Google Chrome is a web browser that is part of the Chromium project, which is an open-source initiative by Google. Amongst the requirements for a valid PWA is the inclusion of a service worker, this is a separate script that must listen for a set of events. Figure 1: Installation prompt on Google Chrome User engagement is why developers need to use a fast and efficient front-end framework to develop their application. If an application takes a long time to load it increases the probability of a bounce. A bounce is defined as a user arriving at a page but then leaves during loading (An, 2018). The front-end of a framework handles the look and behaviour of the user interface, which in turn determines the user experience. 1
Both Blazor Wasm and ReactJS are front-end frameworks, making user engagement relevant for this paper. WebAssembly and JavaScript are standards in programming for web browsers. Blazor Wasm executes its source code by using a runtime written in WebAssembly. The Blazor Wasm runtime is interpreted by the browser. ReactJS is written in JavaScript and its source code is directly interpreted. Both Blazor Wasm and ReactJS provide a programming interface for the developer to alter the user interface. The interfaces achieve this by interacting with the Document Object Model. The Document Object Model (DOM) is a programming interface used to alter Hypertext Markup Language (HTML) documents as nodes in a tree structure. HTML documents along with additional styling represent the user interface on all web applications. The goal of DOM-altering interfaces found in Blazor Wasm and ReactJS is to streamline user interface development by providing generalised code syntax and auto-generation of DOM instructions. Auto-generation of DOM instructions are done using the Virtual DOM (VDOM) technique. When using VDOM, a light copy of the entire DOM is saved in memory to compare with the real DOM. Both frameworks use their own implementation of this technique. With WebAssembly being used for generating layouts for the DOM it is interesting to investigate what performance difference there is in comparison to established JavaScript frameworks. 1.2 Problem statement In this section, the problem is outlined with a smaller background supported by the state of the art. The presented problems regard the lack of PWA-centric frameworks and undocumented front-end performance of the Blazor WebAssembly framework. Developers at Google are developing additions to the Chromium project to further support PWAs running on the browser (Chrome Team, 2021) and argues that applications that have been re-developed to a PWA have a decreased bounce-rate (Google Chrome Developers, 2018). This is due to faster loading times and smaller file sizes when compared to the application before re-development. PWAs occupies a small portion of all web applications on the web. However, there is a steady increase in registered service workers (HM, 2020) indicating growth in PWAs. Hence, there is an indication of increasing adoption and support for PWAs. With the increasing adoption and support of PWAs, there are few indicators of which frameworks are best suited for PWA development. Along with Blazor Wasm being a new release, it is not documented how well it performs on the front-end in a PWA environment. 2
1.2.1 PWA and Web Frameworks Biørn-Hansen et al. (2018b) propose that more web frameworks should be evaluated for PWA- and cross-platform development. For this study selected frameworks are only evaluated for PWA development. In the case of other methodologies, there are technical frameworks created with the methodology in mind. Biørn-Hansen et al. (2018a) conducted a taxonomy and survey of cross-platform frameworks where they connected the hybrid-, interpreted and model-driven approach to other frameworks. Unfortunately, they could not relate the PWA methodology to any framework. 1.2.2 Layout Generation in a PWA Blazor Wasm utilizes a runtime written in WebAssembly to execute its source code. The WebAssembly standard has been evaluated by other papers regarding execution times. However, the WebAssembly applications they have compiled from have been written in a low-level language. In a paper by Haas, et al. (2017) they conducted a benchmark on WebAssembly and JavaScript applications. It was found that web applications written in WebAssembly have faster execution times on mathematical equations in comparison to those written in JavaScript. Cross-compiling JavaScript to WebAssembly has shown speed improvements in executing heavy calculations in the Reiser and Bläser paper (2017). One caveat with their method is that they rely on the developer to write safe code that JavaScript interpreters otherwise protect from. WebAssembly has an architectural difference where it is interpreted before running the application. JavaScript is interpreted during the execution of the application and is presented as a bottleneck in Price and Eberhardt (2018) white paper regarding WebAssembly. WebAssembly applications have faster execution when compiled from lower-level languages. It remains untested how fast a WebAssembly application is when compiled from a high-level coding language. This is what Haas, et al. (2017) suggests for future research in their paper. WebAssembly has been measured to garner a faster web application when compared to JavaScript, but the standard has not been measured in how it can change VDOM generation times. While the previously mentioned studies focused on execution speed, little has been said in academia about the memory allocation differences. Krause (2018) created an interactive benchmark where he evaluated a collection of JavaScript and WebAssembly front-end frameworks in memory allocation. However, Blazor Wasm has yet to be evaluated. 3
1.2.3 Why ReactJS Is Selected For Comparison Today web developers use JavaScript frameworks including Angular, ReactJS, and JQuery (stylised jQuery) to develop most web applications where jQuery is among the most used web frameworks. But there is an ongoing trend shift in popularity where jQuery is losing popularity to ReactJS and Angular (Stack Overflow, 2020). ReactJS is used for PWA development in two previous studies. One being in the study conducted by Biørn-Hansen et al. (2018b) when comparing PWA against applications developed using other methodologies. The second being the application in Huynh and Nguyen-Ky (2020) case study. The DOM interface of ReactJS is measured as the fastest among other existing JavaScript frameworks when altering the DOM using a framework (Persson, 2020). Its popularity, usage in academia researching PWA and its performance in VDOM generation are considered when being selected as a comparative baseline. 1.3 Purpose and research questions With PWA not having a specific framework entitled to it when compared to other methodologies, we can draw from the problem statement that an investigation is needed. Papers by Haas et al. (2017), Reiser, Bläser (2018), and Wang et al. (2019) compared perceived performance between WebAssembly and JavaScript. However, none have used a high-level language as source code for the WebAssembly implementation. Nor did the WebAssembly applications evaluated in the previous papers, use a runtime as Blazor Wasm does. ReactJS has previously been benchmarked in both its performance and DOM-capabilities by Krause (2018) and Persson (2020), but not Blazor WebAssembly. This indicates that there is little known about Blazor Wasm’s performance when generating layouts and what difference a WebAssembly framework can provide in that aspect. Summing up the two problem statements, a purpose is formed: To evaluate the differences that the Blazor Wasm framework can yield for PWA front- end development compared to the ReactJS framework. Using the results obtained from this research we can conclude how a WebAssembly framework impacts the PWA front-end. We can also provide general indicators for which framework, between ReactJS and Blazor Wasm, to select when developing PWAs. This indication is limited to provide documentation and performance when handling changes in the user interface, further limitations are provided in 1.4 Scope and limitations. 4
1.3.1 Presentation of Research Questions To measure how the Blazor Wasm framework supports PWA development entails the following question: [1] To what degree does Blazor Wasm support PWA development in comparison to ReactJS? For a PWA to be reliable according to Richard and LePage (2020) it needs to be performant. To understand the differences in front-end performance the following questions arises: [2] How does a Blazor Wasm application differentiate itself from established JavaScript frameworks in VDOM generation performance? [2.1] How much time does it take to generate a VDOM tree between the two frameworks? [2.2] What are the size differences of VDOM trees in memory between Blazor Wasm and ReactJS? [2.3] How consistent is each VDOM implementation in generation times? 1.4 Scope and limitations Both research questions are limited to the frameworks Blazor Wasm and its baseline, ReactJS. Additionally, the study is conducted within the research area of PWA development. As RQ 1 states, this paper investigates the amount of PWA support provided in the Blazor Wasm framework. Support is defined by how thoroughly PWA subjects are documented. Through a literature study, Blazor Wasm and ReactJS support of PWA development is evaluated. This is done by using the official documentation and YouTube channels for respective frameworks. In this study, PWA support is defined by and limited to the following factors: ● Documentation covering PWA development in relation to the framework ● PWA specific features embedded in the framework ● External PWA tools provided by the framework organisation Programming language for each framework is not put under scrutinization in this study, this is to focus on the implementation of each framework. For RQ 2, the different approaches for VDOM generation are evaluated. This is achieved through experimentation on two PWAs, one created with Blazor Wasm and the other with ReactJS. 5
With speed being part of the reliability factor in a PWA (Richard & LePage, 2020), two factors regarding loading times are regarded when comparing VDOM implementations. Memory usage is investigated to scrutinise other aspects of resource efficiency. For the second research question, differentiation of VDOM is defined by three parameters: ● How fast the framework can generate a VDOM tree ● Space in memory used to store the VDOM tree ● Loading time consistency The lack of time for testing and experimenting on a wider scope is a limitation of this study. Aspects covered by the wider scope include testing the experiment on multiple web browsers and using other popular front-end frameworks like jQuery or Angular. 1.5 Disposition Chapter 1 Introduction, this chapter introduces the research area and presents the gap that is being researched. The research questions are presented, and their selection is motivated. Further, the scope of the problem area and its limitations are also defined. Chapter 2 Method and Implementation, this chapter explains the chosen method for experimentation and the motivation for its selection. This chapter also presents the chosen method for the collection and analysis of the data while also evaluating its validity and reliability. Chapter 3 Theoretical Framework, this chapter presents the state-of-the-art theories and methodologies that are scrutinized. Also, further theories and sources are presented to aid in the understanding of the research questions. This chapter allows the reader to get a deeper understanding of the elements in the research. Chapter 4 Results, the obtained results from the experiment are presented and analysed in this chapter. The results’ relevance to the research questions is presented and discussed. Further, conclusions are drawn based on the obtained results which answer the research questions. Chapter 5 Discussion, in this chapter the choice of method and obtained results are discussed. The chapter presents and discusses the limitations of the study and suggestions for improvements. Chapter 6 Conclusions and further research, this chapter gives proposals for prospects in research work and presents the conclusions that have been drawn from the study. 6
2 Method and Implementation This section describes two methods used in this study. Each method description is divided into sub-chapters based on the research question it aims to answer with the prefix RQ [Number]. The research questions are answered by conducting a combination of a literature study and an empirical study. By conducting a literature study, data and information is collected from various sources and analysed to provide the answer for RQ 1. The empirical study includes an experiment that is studied along with the results it generates. As part of the experiment, two PWAs are created, where one is implemented using Blazor Wasm and the other using ReactJS. The obtained results provide the answers for RQ 2 and its sub-questions. 2.1 RQ 1 - Literature Study This literature study aims to answer research question 1 (RQ [1]) with a qualitative approach. [1] To what degree does Blazor Wasm support PWA development in comparison to ReactJS? This theoretical literature study is focused on finding sources covering PWA subjects and text regarding the three aspects of PWA support. The PWA subjects are defined by aspects presented in the framework that are coupled with the PWA methodology. The keywords used are outlined in 2.4.3 Collecting Information for the Literature Study. A set of sources are used to get relevant literature covering each framework. The sources are maintained by teams and community members working on the framework. Also, each source is either hosted or endorsed as official by each respective framework organisation. The sources for investigating Blazor Wasm are: ● Official Blazor documentation ● The dotNet YouTube channel And the sources for investigating ReactJS are: ● ReactJS Documentation on Create React App ● Facebook Developers YouTube channel Collected information is categorised on the PWA subject it covers. With this information, a summary is created for each framework where a comparison is made regarding the extent of documentation made on selected topics. 7
2.2 RQ 2 - Experiment Design and Execution This method aims to answer research question 2 (RQ [2]) and its subset of questions: [2] How does a Blazor Wasm application differentiate itself from established JavaScript frameworks in VDOM generation performance? [2.1] How much time does it take to generate a VDOM tree between the two frameworks? [2.2] What are the size differences of VDOM trees in memory between Blazor Wasm and ReactJS? [2.3] How consistent is each VDOM implementation in generation times? The method is an experimental simulation as defined by Felderer and Guilherme Horta (2020) where it is conducted in a controlled environment with some realistic factors involved in the experiment environment (p. 41). Two PWAs are developed in Blazor Wasm and ReactJS for the experiment. Both are built following the installable requirements set by Chrome. The main activity of this experiment is to trigger layout generation from each front- end framework. By altering the DOM, the framework is forced to generate a new, or alter the current VDOM tree and apply those changes through the instructions provided by its differentiation algorithm. Each PWA initially creates a set of nodes in a tree structure. Every node contains a numerical value that is altered continuously. Every node is removed when the instance is finished. A flowchart visualising the order and what actions the experiment application executes is seen in Figure 2. 8
Figure 2: Flowchart of the experiment application The experiment is split up into three sessions. Each session has a different number of initial node pairs ( ). See Table 1 for the distribution of node pairs for each session. Notice that the max amount is only lowered from the highest value of 720 pairs. Limitations in the number of elements and the node structure are kept under the Lighthouse DOM size limitations as defined in Web.dev (2019). This is to retain realism in the experiment. 9
Session name npairs Session 1 720 Session 2 360 Session 3 180 Table 1: Experiment sessions and the amount of max elements The activities conducted in the experiment applications are inspired by Persson’s (2020) experiment on evaluating ReactJS VDOM and Procida’s ReactJS Fiber demo (2017). The experiment application executes the following tests in the given order: 1. elements of type are created. Each have a element resulting in 2 elements in total. The elements are distributed by the following rules: ○ Each and pair is distributed so that there is a maximum number of pairs ( ). ○ The element tree is 32 elements deep. ○ Every node that is not on the 32nd level of depth has between 1 - 60 child nodes. 2. For 30 seconds, data within all elements are changed within a repeating time interval of one second. 3. All elements created in step 1 are de-referenced. An overview of how the element tree is structured is viewed in Figure 3. In the figure, coloured nodes are elements while uncoloured nodes are inner data that are altered during the experiment. 10
Figure 3: DOM element structure of the tree The measurement starts when the application is fully loaded, and the start button is pressed on the interface. The application proceeds to alter the state of the page at a one-second interval without input from the user. The Blazor application and the ReactJS application are measured in separate sessions. When changing data, the application invokes the JS / CSS > Style > Layout > Paint > Composite cycle according to the pixel pipeline. Read more about the pixel pipeline in 2.2.3 Measuring Performance in Chrome DevTools. Independent variables of the experiment are: ● Element amount ● Testing time seconds ● Maximum tree depth ● Maximum child amount per node Dependent variables are modified by the independent variables: ● Partition of total time spent on loading ● Tree size in memory ● VDOM generation times 11
Intersecting variables between the independent and dependent variables are for the ● Blazor Wasm PWA ○ Blazor Wasm implementation ○ .NET ○ Mono WebAssembly runtime ● ReactJS PWA ○ ReactJS implementation ● Both implementations ○ JavaScript engine interpreting front-end code Gathered data after an analysis is used to answer RQ 2, RQ 2.1, RQ 2.2 and RQ 2.3. 2.2.1 Implementation details A recursive method is used to generate the DOM tree where an add node function is executed to create a root node. The add function calls itself to add child nodes to the parent node and thereby creates a recursive chain of calls. The independent variables are represented by parameter constants in the source code. To change the values of these variables a redeployment of the PWAs is required. Parameter names are the same between the two projects and are mapped to the independent variables as shown in Table 2. Parameter constant name Represents independent variable Pair amount (npairs) PARAMETER_CONTENT_ELEMENT_AMOUNT PARAMETER_MAXIMUM_TREE_DEPTH_AMOUNT Maximum tree depth PARAMETER_MAXIMUM_CHILD_NODE_AMOUNT Maximum child amount per node PARAMETER_EXPERIMENT_TIME_SECONDS Testing time seconds Table 2: How constant variables in the source code are mapped to independent variables The Blazor Wasm and ReactJS source code can be seen in Figure 4 and Figure 5. DistributableElementsAmount is a global variable that determines how many element pairs can be created and its initial value is set to the parameter constant representing element amount. 12
NodeInstance AddElement(int data, int treeDepth) { NodeInstance element = new NodeInstance(data); DistributableElementsAmount -= 1; if (treeDepth > 0) { for (int i = 0; i < PARAMETER_MAXIMUM_CHILD_NODE_AMOUNT; i++) { if (DistributableElementsAmount > 0) { element.ChildNodes.Add(AddElement(data + 1, treeDepth - 1)); } else { break; } } } return element; } Figure 4: AddElement method in Blazor Wasm 13
function addElement(data, treeDepth) { let elementToBeAdded = new NodeInstance( data, DistributableElementsAmount ) DistributableElementsAmount-- if(treeDepth > 0) { for(let i = 0; i < Parameters.PARAMETER_MAXIMUM_CHILD_NODE_AMOUNT; i++) { if(DistributableElementsAmount > 0) { elementToBeAdded.addChild( addElement(data + 1, treeDepth - 1) ) } else { break } } } return elementToBeAdded.toObject() } Figure 5: addElement method in ReactJS To modify data on the tree, another recursive method is used. The data updating function called ScrambleTree takes a node as an argument, changes its data by incrementing the data by one, and then invokes its own function to modify its child node or nodes. Figure 6 and Figure 7 presents how the ScrambleTree method is implemented in each framework. 14
void ScrambleTree(NodeInstance rootNode) { rootNode.AddToData(1); if(rootNode.ChildNodes.Count
function within a specified interval, for this experiment it is set to a one-second interval. The main difference between the functions is the return statement. In the ReactJS implementation, the method returns an instance to the parent node when reaching the bottom of the tree. While the Blazor Wasm implementation does not return an instance. This is because it was found during the study that when calling methods with a class parameter within the interval invocation method, it takes the class instance as a reference and thereby changes the value using a reference instead of returning an instance. 2.2.2 Specifications The experiment is conducted on a personal computer. The specifications of this computer are: ● Operating System - Windows 10 Education Edition 64 bit ● CPU - AMD A10 PRO-7350B R6, 10 Compute Cores 4C + 6G (4 CPUs), ~2,1GHz ● RAM - 8 GB (6,95 GB Usable) The ReactJS project is dependent on the following React-centric scripts: ● React – Compatible with version 17.0.2 ● React-dom – Compatible with version 17.0.2 ● React-scripts – Version 4.0.3 The Blazor project is compiled using .NET runtime version 4.0.50524.0 in interpreted mode, the experiment is run as a PWA on Chrome Version 90.0.4430.93 and both PWAs are served via the Azure platform as static web pages. 2.2.3 Measuring Performance in Chrome DevTools To measure and evaluate tree generation time, Chrome DevTools is used. The tool analyses performance and accessibility for web applications. The measurement tool is placed in the “Performance” section of the DevTools tab. This tool measures according to the pixel pipeline. The pixel pipeline is a chronological order of steps executed to load a page. It is divided into three portions. These portions are JavaScript and two rendering portions where the JavaScript portion is first (Lewis, n.d.). The order can be seen in Figure 8. In this study, the JavaScript portion is analysed. 16
Figure 8: Lewis, P. (2019). JS / CSS > Style > Layout > Paint > Composite [Figure]. Retrieved 20 April 2021, from https://developers.google.com/web/fundamentals/performance/rendering/ It is during the JavaScript portion of the pixel pipeline in which Blazor and ReactJS generate their layouts. By measuring the microseconds to execute the JavaScript portion (Jduration) it is possible to gather the time it takes to generate a VDOM Tree. Then by gathering the difference in memory usage between the two points of starting (Jstart) and finishing (Jstart + Jduration) the VDOM-tree size is gathered. The event that invokes the JavaScript portion is named TimerFire and is searched for when analysing the profiling files. TimerFire is a complete event and is categorised as a phase-type by the letter X. 2.3 Method Motivation The chosen method consists of a combination of a literature study and an empirical study. This gives a qualitative and quantitative approach to the purpose of the study. RQ 1 is approached by analysing the official documentation of both frameworks as well as videos found on official YouTube channels. The purpose of this is to understand the relation of the PWA with the frameworks and how well each framework supports the PWA methodology. Studies indicate that 18-22 % of developers, regardless of experience, prefer watching videos as a source for learning new skills (DevelopIntelligence, 2018). Hence, the official YouTube channels belonging to the two companies have been selected as sources for information collection. In the empirical study, the tests are designed to understand the page loading times for each application. This provides results that indicate whether there is a difference in page loading times which answers RQ 2. 2.4 Data collection The data gathered from the literature study is selected depending on its relevance to the research area and more specifically how well it provides the answer for RQ 1. 2.4.1 Gathering Performance Profiles for the Experiment Google DevTools for Chrome is used to record performance variables. Google DevTools is fitting for the measurement since it is integrated within the test 17
environment and provides the needed variables. The toolset records according to the pixel pipeline, meaning it separates rendering tasks from invoked JavaScript methods. Data is collected by creating a snapshot recording called a profile. The creation is initialised from the performance tab of DevTools. For this experiment the main thread is recorded, this is because Blazor’s and ReactJS’ runtime is run on the main thread only. When the website layout changes, the following steps according to the pixel pipeline is executed: JS > Style > Layout > Paint > Composite The measurement only collects data for the JavaScript portion. From the gathered profile a set of data is selected. The selected variables are: ● Time to generate layout ● Tree size in memory ● The portion of total time used to generate layout Time until re-rendering is the time used to execute the JavaScript portion of the pixel pipeline, this is selected to measure the time of generating a VDOM tree. This is saved in microseconds. Data regarding memory usage is collected due to it representing how much space a VDOM tree takes in memory. The VDOM tree size is collected as bytes. Using a selective approach, the data points are picked. Data is only collected when a layout change is invoked. This is done to avoid collecting irrelevant data. A sample of 30 measurements from each implementation is collected. Each sample has its own set of data points. The portion of total time used to generate a layout is the fraction of the total time used to generate one VDOM tree. This sample's standard deviation is used to measure the consistency of each VDOM generation technique. The fraction is a value between zero and one. Every profile is saved as a JavaScript Object Notation (JSON) file for a separate analysis session. The JSON file type is a structured type that uses keys and associated collections to identify data. It is also widely accepted and can easily be read by other applications. 2.4.2 How the Experiment Is Automated The experiment is conducted using an automatization script. This script is written specifically for this study. It is a Python 3.8.2 script for 32-bit systems. The script takes control of the computer interface by managing the computer mouse and keyboard to forward the experiment. 18
The library PyAutoGUI (PAG) is used for mouse, keyboard, and user-interface control. Along with PAG, the script uses standard libraries included in the current Python version which is the Time- and OS library. OS is used for starting the PWA and Time is used to get timestamps when the experiment is done. Before starting the script, it must be provided with the following values and resources: ● App-id – Unique identification string provided for specific PWA ● Window title – Title of the application shell window on the interface ● Reference image of the start button – A image of the start button seen on the user interface ● Button image name – Name of the reference image file used The app-id parameter is a unique identification string for an installed PWA. The app- id string is retrieved by inspecting the desktop shortcut created by Chrome when installing the PWA. The window title is a string that represents the title of a process window on the operating system. A reference image and the name of the reference image need to be supplemented for the script to find the start button in the application interface. The reference image is a screen capture of the button element when the PWA window is maximised. When the script has reached its done state, it saves the profile and finishes the instance. 2.4.3 Collecting Information for the Literature Study The collection of information is done through a literature study involving official sources that belong to the two companies. Articles and videos from each source are searched for using the search functionalities found within each website. The terms used for searching articles and videos are “PWA” and “Progressive Web Application”. 19
Keywords are used in searching through selected sources. The keywords are related to features that are specific to PWAs. The keywords are based on conceptual terms and features found during the initial literature search. These being terms from Altexsoft (2018), terms from Mozilla (2021), and features showcased in Whatpwacando (Moerkerke, n.d.). The selected keywords are: ● Offline mode ○ Caching pattern ○ IndexDB ○ LocalStorage ● Push notifications ● Service worker ● Manifest ● Hardware access terms ○ Camera ○ Microphone ○ Geolocation ○ Vibration (Mobile) ○ Device orientation and motion On studying the documentation of each framework, the differences are sought after and noted. The amount of PWA subjects covered in the documentation indicates the degree to which support is provided by the framework. In the literature study, it is also sought after the PWA specific features embedded in each of the frameworks. Furthermore, it is also studied if the framework companies provide external tools that aid in the development of PWAs. These observations along with the documentation content collected and presented in the upcoming chapter. Thereafter, the data can be analysed to draw conclusions and provide the answer for RQ 1. 2.5 Data analysis The authors explain the Z-test and standard deviation equations used on gathered data from the experiment. Also, how search words are used in the literature study and utilised in the selection of literature data. 2.5.1 Experiment - Comparing Means of Independent Samples There are two sets of samples, the Blazor Wasm sample from the Blazor Wasm population and the ReactJS sample from the ReactJS population. For every retrieved variable in each sample, a standard deviation and mean are calculated. Tree generation times, tree size and time spent on generating during layout 20
generation have their own mean and standard deviation calculated. Standard deviation is used to understand the consistency of the dependent variables. The equation for standard deviation is defined in Equation 1. ∑ 2 − (∑ )2 = √ ( − 1) Equation 1: Standard deviation n = total amount of data x = data point value from s = resulting standard points sample deviation The standard deviation is used to conduct a Z-test. A two-sample Z-test provides the differentiation between the populations (µR and µB) using mean values and standard deviation between the two samples. When performing the Z-test, the following hypotheses are tested: Null hypothesis (H0): Blazor Wasm performs near equally as well as its ReactJS counterpart (µR - µB = 0). Alternative two-tailed hypothesis (H1): There is a difference in performance between the Blazor Wasm- and ReactJS PWA (µR - µB ≠ 0). H0 and H1 is a question regarding equality between the samples from each population, hence making this a two-tailed test. The Z-value equation is defined in Equation 2. ̅̅̅̅̅̅̅̅̅ − ̅̅̅̅̅̅̅̅ = 2 2 √ + Equation 2: Z-value z = z-value ̅ = amount of data points in the sample The confidence interval (α) is set to 0,05. Meaning that the experiment is analysed with 95% confidence that the results are correct. The critical value (p) based on the selected confidence interval is approximately 1,95996. If z > p or z < -p then the null hypothesis is rejected, and it is reasonable to suggest a difference in the tested variable between the two samples. If proven otherwise, there is no significant difference between the populations. 21
2.5.2 Literature Study PWA subjects are searched for within the content of the gathered data. Gathered data are categorised by its source and what subject it covers. If the selected data point does not contain any keyword from the list of subjects, then it is marked as missing. For each framework these three aspects are summarized: ● Documentation covering PWA in relation to the framework ● PWA specific features embedded in the framework ● External PWA tools provided by the framework organisation 2.6 Validity and reliability Various cautions have been taken during the empirical study as well as the literature study to increase the validity and reliability of the research. Examples of these precautions are multiple iterations when conducting the empirical study and using reliable and credible sources for the literature review. 2.6.1 Validity The memory difference gathered during the experiment is used to analyse the size of the generated DOM tree, this is valid because VDOM trees are stored in memory. The two sets of values for memory usage and page loading are utilised to formulate an answer for RQ 2. By measuring the time and resources it takes to execute the JavaScript portion upon altering the layout data for each framework RQ 2 and its sub-questions can be answered. RQ 2 is a descriptive question where it mainly requires quantitative data to be fully answered, therefore a dataset with multiple data points containing layout generation time and heap memory changes suffice. The literature search is done by gathering data for RQ 1. With qualitative knowledge of the source, it is possible to conclude with a descriptive analysis of how each framework supports PWA development in the selected factors detailed in 1.4 Scope and limitations. 2.6.2 Reliability Google DevTools has its source code open for sharing and reworking with its source code, as of writing, is updated frequently. It is developed by the Google Chrome team and is fully integrated with the Chrome web browser. The collection of data is done by reading performance measurement files provided by the development tools in Chrome. A Python script is used to automate the data collection process. 22
Data files are read by another Python script that picks samples by category and function name. With collected data, the script calculates z-value, critical value, and standard deviation using the two samples. The statistics module of the SciPy library is used to gather the critical value according to the z-table. The reason for automating the collection and calculation of data is to decrease human error and time consumption. Moreover, the experiment is designed to prevent any of the frameworks from having an advantage over the other. Both applications perform the same tasks and are tested using the same utilities, such as the same computer and the same version of Chrome. Also, background applications are stopped to limit their impact on the experiment results. To keep likeness in both implementations, the code is written using built-in methods and primitives with the same functionality if there is no other preferred way in the respective language. By the same function meaning characteristics and implementations that are alike. Structure and the variable names of the source code are written alike. See attached code in 2.2.1 Implementation details and 8 Appendixes for practical examples. Blazor Wasm is only compared with ReactJS for each research question and method in this study. This is done to keep results consistent. To further increase the reliability of the experimentation and allow for future research, the PWAs and automatization scripts are published on a public GitHub repository. The experiment is conducted 30 times per implementation to ensure that the obtained population is not impacted by interfering environmental factors, like available computer resources, automatic background tasks etc. This is done along with the analysis to gain a plausible statistical significance. Furthermore, the literature study is based on reliable sources of information. To gather information and conduct the literature study, sources are critically reviewed. The selected sources are close to each framework and include minimal irrelevant information regarding each framework. 2.7 Considerations In this section, the considerations of this study are outlined. Considerations regard the selection of information resources for both literature studies conducted in this study. Moreover, how source code is written in each experimental application to ensure a fair measurement is outlined. 23
2.7.1 Selection of Sources Numerous different sources are used to conduct the preliminary literature study to understand the research area. Hence, critically reviewing these sources and their credibility is an important consideration. To ensure that sources for the literature study method are reliable, only official sources belonging to the respective company have been used for gathering information. Some of these sources refer to other external sources, also these are critically reviewed during the study. Since videos are a part of the literature study, the content makers, as well as the contributors in the videos, are assessed to ensure their knowledge and background. Moreover, throughout the research, referencing correctly, fairly, and ethically has been taken into consideration. 2.7.2 Implementation Fairness The scientific aspects that have been taken into consideration are to design the experiment to be fair to both frameworks. The source code of the two PWAs is written to minimize the risk of giving one advantage over the other. The experiment is conducted on the same computer and uses the same version of Chrome. During the experimentation, it is ensured to stop non-vital applications running in the background that can interfere with the results of the experiment. Code style in both projects is selected to fit each programming language which considers the placement of curly brackets and indentation styles. Therefore, the placement of the curly brackets is different between the NodeInstance class declarations as seen in Figure 9 and Figure 10. 24
class NodeInstance { constructor(data, id) { this.data = Number(data) this.childNodes = Array(0) this.id = Number(id) } toObject() { return { data : this.data, childNodes : this.childNodes, id : this.id } } addChild(element) { this.childNodes.push(element) } addToData(addition) { this.data += Number(addition) } } export default NodeInstance Figure 9: Class declaration in NodeInstance.js 25
using System; using System.Collections.Generic; namespace pwa_blazor.Models { public class NodeInstance { public int Data { get; set; } public List ChildNodes { get; set; } public NodeInstance(int data) { Data = data; ChildNodes = new List(); } public void AddChild(int data) { ChildNodes.Add(new NodeInstance(data)); } public void AddToData(int amount) { Data += amount; } } } Figure 10: Class declaration in NodeInstance.cs 26
3 Theoretical framework This section is populated with detailed background information and theory about the different components included in this study. The research questions are related to the state of the art found within each respective area. The structure of this section is a research question followed by the technical components of the state of the art. Blazor Wasm and ReactJS are frameworks that can be utilized for the creation of PWAs, thus certain aspects of the frameworks are chosen to be evaluated in this research. 3.1 Research Questions Related to the State of the Art RQ [1] To what degree does Blazor Wasm support PWA development in comparison to ReactJS? Blazor Wasm had its first preview release in 2019 as a part of ASP .NET Core 3.0 and its first stable release on the 19th of May 2020 (Roth, 2020a). This indicates that Blazor Wasm is a relatively new framework. Included with Blazor Wasm’s first preview release was a PWA template project, entailing its intended use as a framework for PWA development. ReactJS is a well-established web framework where its first public version came out on the 2nd of July 2013 on their GitHub repository (O'Shannessy, 2013). ReactJS has been used to create a PWA in Biørn-Hansen et al. (2018b) evaluation of cross- platform methodologies. Huynh and Nguyen-Ky (2020) conducted a case study on a PWA developed in ReactJS. The study focused on web-based applications for facility management. In this study, they claimed that ReactJS was a good choice for PWA development. Like Biørn-Hansen et al. (2018b) they considered the PWA methodology as a platform unifier. Mena et al. (2019) implemented a PWA gathering geospatial data. In their preliminary literature study, they looked at PWA features supported in the Chrome browser on different platforms. It is not defined in their report which front-end framework they used for their PWA. On inspecting their deployed demo GeoNat (Mena et al., n.d.), it appears as they opted for AngularJS as a front-end framework for their PWA. They concluded that the development of a PWA made it easy to broaden the platform compatibility. But they did not detail why they selected AngularJS as the front-end framework for their PWA. 27
[2] How does a Blazor Wasm application differentiate itself from established JavaScript frameworks in VDOM generation performance? [2.1] How much time does it take to generate a VDOM tree between the two frameworks? [2.2] What are the size differences of VDOM trees in memory between Blazor Wasm and ReactJS? [2.3] How consistent is each VDOM implementation in generation times? WebAssembly has been measured as being consistent in performance. Reiser and Bläser (2017) garnered a decrease in execution time in their Speedy.js project. In their study, the label decrease in execution time as “speedup” and garnered ranges from 2% to 364% speedup in 14/16 cases. Regarding the remaining two cases, one had the same speed as its TypeScript source and the other suffered a -2% speedup metric. All sixteen cases consisted of eight TypeScript scripts compiled to WebAssembly running on two browsers each. Selected browsers for their empirical study were Chrome 59.0.3071.86 and Firefox 53.0. Haas et al. (2017) measured a 37% faster average when running C programs compiled to WebAssembly bytecode. This was in comparison with C programs compiled to asm.js code. The applications were part of the PolyBenchC suite, which is a collection of performance heavy calculations. Wang et al. (2019) had a similar method to Reiser & Bläser (2017) where they used WebAssembly as a compilation target for JavaScript. However, the goal of Wang et al. (2019) was to obfuscate the JavaScript source code by compiling the JavaScript to WebAssembly and obfuscating the generated byte code. On comparing obfuscated code by JSPro against other commercial tools they found a range of 15% - 32.8% reduction on executing time. In Krause's (2018) benchmark he measured memory usage in ReactJS 16.4.1, Yew and StdWeb. Yew and StdWeb are, like Blazor Wasm, front-end frameworks running on WebAssembly. For the benchmark, the applications conducted the following actions: ● Created 1000 rows in a table right after the page loaded (ready memory) ● Updated every tenth row five times (run memory) ● Replaced all rows five times ● Replaced and recreated all rows five times In Table 3 the memory usage (memory amount +- standard deviation) after a certain action is presented. Worth noting is that Yew is non-dependant on keys, this means that it can reuse existing DOM elements without having to instantiate new ones. 28
Action name Yew 0.5.0 StdWeb 0.4.8 React 16.4.1 Initial memory 2,5 +- 0,1 2,5 + − 0,2 2,8 + − 0,2 Create 1000 rows 4,7 +- 0,0 3,1 + − 0,0 6,7 + − 0,0 in a table Update each 10th 5,4 +- 0,0 3,1 + − 0,0 7,6 + − 0,0 row for all rows five times Replace all rows 5,3 +- 0,0 4,3 + − 0,2 7,9 + − 0,0 five times Creating and then 14,9 +- 0,2 3,6 + − 0,1 3,8 + − 0,0 clearing all rows five times Table 3: Memory usage +- standard deviation in Megabytes. Adapted from: Krause, S. (2018, September 27). Results for JS web frameworks benchmark – round 8. Stefan_Krause.blog(). Retrieved May 16, 2021, from https://stefankrause.net/js- frameworks-benchmark8/table.html 3.2 Technical Components of the Study In this section, the technical components and methodologies used in the study are explained. The explained items are PWA, WebAssembly, Virtual DOM, Blazor Wasm and ReactJS. 3.2.1 Progressive Web Application Progressive Web Application (PWA) is characterized as a type of software that is executed in a dedicated web browser using web- and native technologies. Browsers dedicated to one site are called a Site-specific browser (SSB). PWAs being run on an SSB provides a similar experience to using native applications (Osmani, 2015). Figure 11 is a comparison between how a PWA is viewed in a normal web browser and an SSB. 29
Figure 11: ReactJS PWA on a Chrome browser window (left) and a Chrome SSB Window (right) Unlike native applications, a PWA is distributed by visiting a website and directly within the interface of the web browser, making the conscious decision to “install” that website. When a PWA is installed, it becomes available to function without an internet connection. Every browser has its own criteria for the installation of PWAs. For a PWA to be installable on a Chrome web browser it must satisfy three requirements (LePage, 2021). The browser looks for these requirements before allowing the user to install the PWA. The requirements are: ● Having a Service Worker listening for fetch events ● Serving the page on an HTTPS connection ● Having a Web app manifest file It was planned to enforce a fourth requirement for Chrome 93, this requirement entailed an offline fall-back page if the device is not connected to the internet. Google has now revoked that requirement for future Chrome releases and put it on hold (McLachlan, 2021). Service workers are JavaScript files run in the background. For a PWA, a Service worker usually handles offline functionality. Another property of a PWA is to have the capability to access the hardware of its host devices such as the camera, microphone, and geolocation. Sending and receiving push notifications is another capability that PWAs have. While these features are not specific to a front-end framework, it is closely tied to PWAs and the near-native experience they can provide (Richard & LePage, 2020). Currently, the development of the PWA front-end is done with either the use of JavaScript frameworks or plain JavaScript. However, the latest implementation of JavaScript is considered insufficient for the development of high-performing web 30
applications (Haas, et al., 2017; Price & Eberhardt, 2018), meaning that there is a need for an alternative that can do heavy computations with minimal overhead. 3.2.2 WebAssembly WebAssembly is a low-level instruction language by the WebAssembly Working Group. The group contains members from 19 organizations including Mozilla, Microsoft, Google, and Apple. The language has been approved as a web standard by the World Wide Web Consortium (W3C). WebAssembly aims to complement JavaScript and fulfil low-level tasks that JavaScript subsidiaries like asm.js have failed to satisfy effectively (Haas, et al., 2017). Due to WebAssembly being a W3C standard, it is supported by Firefox, Safari, and Chromium browsers (Edge, Chrome, Opera, etc.). WebAssembly is partly designed to be a compilation target, making it possible to write applications in programming languages other than JavaScript and make them executable in the browser. JavaScript and WebAssembly use the same interpreter engine when executed. What differentiates the two are the steps that are taken to parse their code to bytecode. Every major interpreter engine handles its source code differently, but Clark (2017) has made a comparison in how SpiderMonkey could run WebAssembly code in comparison to JavaScript. JavaScript has two extra steps in its parsing chain, these steps are garbage collection and re-optimising. Microsoft introduced the Blazor WebAssembly framework to replace JavaScript front-end code with their own .NET platform (Microsoft, n.d.). 3.2.3 Virtual DOM Before describing the technical concepts around Blazor WebAssembly, one of the major patterns needs to be outlined. Namely the Virtual DOM. To alter the layout of a page during runtime, it is required to use the DOM API found in the web standard. But implementing efficient JavaScript that alters DOM efficiently takes development time and requires very specific layout-altering. To simplify front-end development a new approach was developed, this pattern is called Virtual DOM (VDOM). Figure 12 presents an overview of how a page layout can be altered using the VDOM approach. 31
Figure 12: Overview of a Virtual DOM when changing an element VDOM is a method that is an in-memory representation of the current state of the page, this is represented as a data tree. When the in-memory representation is changed a new tree is generated. A differentiation algorithm (or diff algorithm) is used between the old and new trees to track which changes have been made to what element. From the diff algorithm, a set of instructions are generated, these are used for executing layout altering functions on the DOM API. 3.2.4 Blazor WebAssembly Blazor WebAssembly (Blazor Wasm) is a Single-Page Application (SPA) framework developed by Microsoft. Blazor Wasm can be used to create SPA with PWA characteristics. 32
The source code in a Blazor Wasm is written in C#. By using a WebAssembly runtime called Mono, a Blazor Wasm application can run its front-end code on the browser. Mono is a .NET runtime and is used by Blazor Wasm to execute compiled .NET code on the web browser. A Blazor Wasm application uses the Mono runtime as a compilation target. Compiling the runtime and letting it interpret the source code during runtime is called Interpreted mode. Meaning that C# code only needs to be compiled for the .NET runtime to work in the browser. An overview of how the architecture of a Blazor Wasm application that is compiled in Interpreted mode is seen in Figure 13. Figure 13: Sanderson, S. (2018). Interpreted Mode Overview [Figure]. Retrieved 15 April 2021, from https://blog.stevensanderson.com/2018/02/06/blazor-intro/ Front-end code in a Blazor Wasm application is characterized by user interface (UI) logic. UI Logic refers to how the state of the interface changes depending on pre- defined logic and user input. A web page that changes its content in this manner is referred to as a dynamic page. Dynamic pages are web pages that change depending on the current state of present data. Normally the state of a dynamic page is determined by the hosting server but in Blazor Wasm’s case, the state is determined by the client. Razor is a markup language that combines HTML syntax with C#, enabling the developer to use C# syntax for generating the template. When using Razor markup in Blazor it binds data with the client, builds the page, and changes the layout depending on logic written in C#. In Blazor Wasm, all pages and page-specific logic are written in Razor markup within component files. To be resource-efficient in altering the page layout, the Blazor Wasm client only updates the elements whose state gets modified. This is done using a VDOM 33
You can also read