Interactive 3D Knowledge Platform for Reconstructive Plastic Surgery
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Självständigt arbete i informationsteknologi 15 juni 2022 Interactive 3D Knowledge Platform for Reconstructive Plastic Surgery Aria Assadi Kevin Hormiz Faizan Mian Oskar Rick Carl Wallskog Civilingenjörsprogrammet i informationsteknologi Master Programme in Computer and Information Engineering
Abstract Institutionen för Interactive 3D Knowledge Platform for Recon- informationsteknologi structive Plastic Surgery Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1, hus 10 Aria Assadi Postadress: Kevin Hormiz Box 337 Faizan Mian 751 05 Uppsala Oskar Rick Carl Wallskog Hemsida: https://www.it.uu.se There is a lack of technical platforms where reconstructive surgeons around the world can share their knowledge and experience. This prob- lem is especially prevalent in developing countries, where surgeons rely on inadequate means for preparing operations. This project is called Lambå-Atlas and addresses this issue by providing the surgeons with an intuitive website. It consists of an interactive human 3D model, where surgical instructions are accessible by clicking on the respective body parts. The purpose of Lambå-Atlas is to assist reconstructive surgeons with suggestions and assessments of suitable surgeries based on the re- quired medical equipment and experience. The goal of this project was to create an educational platform that promotes knowledge exchange. Lambå-Atlas has achieved this by providing a website that aids recon- structive surgeons by letting them navigate a 3D human model and sug- gest suitable surgery instructions based on a chosen body part. Extern handledare: Dr. Olof Engström och Dr. Morten Kildal vid Akademiska sjukhuset Handledare: Mats Daniels, Martin Eriksson, Björn Victor och Tina Vrieler Examinator: Björn Victor
Sammanfattning Det finns en brist på tekniska plattformar där rekonstruktionskirurger runt världen kan dela med sig av sin kunskap och erfarenhet. Detta problem är allmänt förekommande i utvecklingsländer där kirurger förlitar sig på inadekvata medel för operationsplanering. Detta projekt kallas Lambå-Atlas och löser problemet genom att förse kirurgerna med en intuitiv hemsida. Den består av en interaktiv mänsklig 3D-modell där tidigare doku- menterade operationer är åtkomliga genom att klicka på de respektive kroppsdelarna. Syftet med Lambå-Atlas är att hjälpa rekonstruktionskirurger hitta lämpliga operationer baserat på deras tillgängliga medicinska utrustning och erfarenhet. Målet med detta pro- jekt är att skapa en utbildningsplattform som främjar utbyte av kunskap. Lambå-Atlas uppnår detta genom att förse kirurger med en hemsida som hjälper dem hitta lämpliga operationsinstruktioner. Genom att navigera en 3D-människomodell kan en kirurg hitta instruktioner baserat på en vald kroppsdel. ii
Contents 1 Introduction 1 2 Background 3 2.1 Uppsala University Hospital Partnership . . . . . . . . . . . . . . . . . 3 2.2 Plastic Surgery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2.1 Flap Surgery . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.3 Access to Electronic Devices in Developing Countries . . . . . . . . . 4 2.4 3D-modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3 Purpose, Aims, and Motivation 5 3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1.1 The Need for Lambå-Atlas . . . . . . . . . . . . . . . . . . . . 5 3.2 Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.2.1 Sustainable Development Goals . . . . . . . . . . . . . . . . . 6 3.3 Ethical Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3.1 Representation on the Main Page Human Body . . . . . . . . . 6 3.3.2 Pictures from Previous Surgeries . . . . . . . . . . . . . . . . . 8 3.3.3 Functionally over Appearance . . . . . . . . . . . . . . . . . . 8 3.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4 Related Work 9 4.1 Similar Purpose and Functionality to Lambå-Atlas . . . . . . . . . . . . 9 4.1.1 Microsurgeon.org . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1.2 Touch Surgery: Surgical Videos . . . . . . . . . . . . . . . . . 10 4.2 Similar Design to Lambå-Atlas . . . . . . . . . . . . . . . . . . . . . . 10 iii
5 Method 10 5.1 Choice of Software Platform . . . . . . . . . . . . . . . . . . . . . . . 10 5.2 Designing a Prototype in Figma . . . . . . . . . . . . . . . . . . . . . 11 5.3 Front-end Programming Language . . . . . . . . . . . . . . . . . . . . 13 5.4 JavaScript Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.5 Handling 3D Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.5.1 3D-file Formats . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.5.2 Modifying 3D Objects . . . . . . . . . . . . . . . . . . . . . . 14 5.6 Building the Back End . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.7 Choice of Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 6 System Structure 16 6.1 Admin and Client Site . . . . . . . . . . . . . . . . . . . . . . . . . . 17 6.1.1 Admin Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 6.2 CRUD and User Permissions . . . . . . . . . . . . . . . . . . . . . . . 17 6.3 Communication Between the Front End and Back End . . . . . . . . . 17 6.4 Server and Database Communication . . . . . . . . . . . . . . . . . . . 18 7 Requirements and Evaluation Methods 18 7.1 User Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.2 Evaluation of User Tests . . . . . . . . . . . . . . . . . . . . . . . . . 19 8 Implementation of the 3D Model 19 8.1 Dividing the 3D Body Into Body Parts . . . . . . . . . . . . . . . . . . 19 8.2 3D-modeling Software . . . . . . . . . . . . . . . . . . . . . . . . . . 20 9 Results and Discussions 21 iv
10 Future Work 22 10.1 Local Data Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 10.2 Storing Patient Information . . . . . . . . . . . . . . . . . . . . . . . . 22 11 Conclusions 23 A Admin Page 29 B User Tests 31 C Survey Results 34 v
1 Introduction 1 Introduction Reconstructive surgery is a procedure for restoring lost or injured body parts or tis- sue [52]. For instance, in a case where the tissue of a hand has been damaged, this type of surgery helps restore its function and aesthetics. Surgeons in the area of reconstruc- tive surgery find it difficult to plan and perform surgeries due to a lack of accessible information. This problem is especially prevalent in developing countries such as Tan- zania, Ethiopia, and Palestine where surgeons have to rely on surgical instructions from YouTube videos to prepare for surgery [41]. In order to solve this problem, a web- site that can assist reconstructive surgeons all around the world in preparation for and execution of surgeries, has been created. Lambå-Atlas is a collaboration between students at Uppsala University and the plastic surgery team at the University Hospital in Uppsala. The word “lambå” is the Swedish word for “flap”, which refers to tissue on the body. Flap surgery is the act of transferring a unit of tissue from one site of a human body to another while maintaining the tissue’s blood supply. The University Hospital has provided learning material with instructions on different reconstructive surgeries, which has been used to create a website where the user can access these. Upon entering the website, the user is presented with a 3D male human model, that can be rotated, zoomed, and clicked. When clicking on a body part, the user will be navigated to a page consisting of instructions for reconstructive surgeries relevant to the chosen body part (see Figure 1). Because the surgeries differ in difficulty and material requirements, the instructions for the surgeries will be sorted by both those require- ments. This is done by giving every instruction two ratings, one for difficulty and one for material requirements. The ratings range from one to three, where a three in diffi- culty represents that the surgery is difficult to perform in regard to surgical experience. A three in material requirement represents that the surgery requires advanced medical equipment. As both ratings descend, the difficulty and material requirements become less advanced. After taking part of the surgical instructions, surgeons can leave a one to five-star rating, depending on the usefulness of these. 1
1 Introduction Figure 1 Lambå-Atlas menu showing surgical instructions for the trunk Although Lambå-Atlas can be used by any reconstructive surgeon, its purpose is to simplify for those surgeons who do not have the option to search for information in medical databases. With the growth of Lambå-Atlas, surgeons across the world will have uniform information about reconstructive surgery. Acknowledgement We would like to express our sincere appreciation to our exter- nal supervisors, Dr. Morten Kildal and Dr. Olof Engström, for the continued support and encouragement throughout the project, as well as providing us with the material needed. We are also grateful for the ability to work on this project, and the learning possibilities provided by Uppsala University Hospital. Declaration of division of labor The division of labor has evenly been distributed in the group. Faizan Mian and Kevin Hormiz have mainly focused on developing the design and front-end code for the website. Carl Wallskog has been responsible for the 3D-modelling of the human body and has also contributed to the development of front-end code for the website. Aria Assadi och Oskar Rick have mainly focused on developing the database and API. The corresponding parts of the report have mostly been written by the respective person. Other parts of the report have been addressed cooperatively by all authors. 2
2 Background 2 Background In this section, a few of the areas relevant to the project will be explained. Firstly, the stakeholder of the project, Uppsala University Hospital, will be introduced with a background as to what they are and how they work. Secondly, reconstructive plastic surgery and its relation to the project will be described. Furthermore, the access to electronic devices in developing countries and the possibility of 3D-modelling for this project, will be discussed. 2.1 Uppsala University Hospital Partnership At the department of plastic surgery at Uppsala University Hospital, surgeons perform reconstructive plastic surgery, which is described as the restoration of a patient’s func- tionality and appearance after tumor disease, severe body injury or malformations [25]. With the help of state-of-the-art machinery, a team of professional surgeons and nurses treat over 150 patients every week [25]. The University Hospital is not only active in Sweden but collaborates with other countries such as Tanzania, Palestine, and Ethiopia. Dr. Morten Kildal, the external supervisor for this project, is part of the team traveling to these countries, educating surgeons on how to perform reconstructive surgery. After visiting and educating surgeons in these countries, Dr. Kildal experienced that surgery preparations were inefficient. The reason for this is that surgeons had to —while search- ing for surgical information— examine material that was spread across books, articles and online videos, making the preparation inefficient. 2.2 Plastic Surgery Plastic surgery is a surgical specialty that includes altering of a person’s appearance, and restoration or reconstruction of facial and body tissues. It can be divided into two categories, namely cosmetic surgery and reconstructive surgery. Cosmetic surgery fo- cus on improving a person’s appearance, while reconstructive surgery is used to recon- struct a body part or improve the function of it. Some types of body abnormalities that reconstructive plastic surgery is used for are caused by, diseases, trauma, tumors and developmental abnormalities [30]. 3
2 Background 2.2.1 Flap Surgery Although reconstructive surgery does not exclusively consist of flap surgery, in this report, both terms are used interchangeably. Flap surgery is namely a technique used in reconstructive surgery, where surgeons try to transport healthy live tissues from one location of the body to another whilst preserving blood supply to the new area. It is practiced in those areas of the body that have lost skin, fat, muscle movement, and/or skeletal support due to severe injuries or diseases [52]. Lambå is the Swedish translation of the term flap, hence creating the origin of the project’s name. The name Lambå-Atlas is referring to the website as an atlas containing a collection of flap surgeries on the human body. 2.3 Access to Electronic Devices in Developing Countries In order for Lambå-Atlas to function all around the world, including developing coun- tries such as Tanzania, Ethiopia and Palestine, a few aspects need to be considered. The access to electronic devices vary in different parts of the world. In general, the access to mobile phones is greater than the access to desktops and tablets [53], something that is especially true in developing countries [54]. In order to create a platform that is accessi- ble in all parts of the world, Lambå-Atlas needs to be available and suitable for mobile phones. 2.4 3D-modelling 3D-modelling is the process of developing an object simulated in 3D-space. This can be traced back to Ancient Greece, where Euclid developed ideas about geometry that in- spired many 3D-modelling techniques. Computer-aided design, more commonly known as CAD, developed in the 1950s, is more in-line with what 3D-modelling is today [56]. 3D-modelling is widely used in movies and video games, and assist with creating com- plex scenes that would be hard to recreate in real life. In addition to this, 3D-modelling is also used in architecture, engineering, and advertisements. 3D-modelling is used with specialized 3D-softwares that is used to create and manipulate points in space, called vertices. A collection of the vertices forms a mesh, that also creates an object [51]. 3D- modelling is important for Lambå-Atlas user interface, a human body, that is created using 3D-modelling. 4
3 Purpose, Aims, and Motivation 3 Purpose, Aims, and Motivation In this section, the importance of a platform for knowledge exchange in the field of reconstructive surgery will be discussed, along with the aims and motivations for this project. Another subject such as ethical aspects is also explored due to its relevancy to our project. The delimitations of the project are also discussed. 3.1 Purpose The purpose of Lambå-Atlas is to reduce the time that reconstructive surgeons spend on searching through books, articles, and videos by gathering information from different surgeries into one platform. Other than reducing the time spent on research before surgeries, the platform will also be helpful while educating new surgeons in the field of reconstructive surgery. In low income countries, where the availability of research can be limited, fewer patients might be treated due to a lack of knowledge [41], something which this project aims to improve. 3.1.1 The Need for Lambå-Atlas In an article written by Frank Fang and Kevin C. Chung, they discuss how flap surgery has evolved throughout history. They state that one of the factors in the development of flap surgery of the upper extremity —i.e., the part of the body that includes the arm, wrist, and hand [43]— is knowledge exchange [32]. By providing a platform such as Lambå-Atlas where reconstructive surgeons around the world can share and access knowledge, we hope to contribute to further development of flap surgery. Fang et al. further state that “trauma generated by the maiming insults of war and industry” (p. 1) is a factor that historically has —because of its complexity— developed the practice of flap surgery for the upper extremity. This shows the necessity of introducing an application like Lambå-Atlas in countries where such injuries are prevalent. 3.2 Aims The website aims to achieve a few long-term goals. One of the main goals is to create a platform that promotes knowledge exchange and education for reconstructive surgeons. Another goal for Lambå-Atlas is that it should contain information suited for all recon- structive surgeons, regardless of their experience or medical equipment. As previously 5
3 Purpose, Aims, and Motivation described in Section 1, a rating system for difficulty and material requirement will be available for all surgical instructions on Lambå-Atlas. If there are enough instructions for every body part where every combination of ratings is covered, this goal will be fulfilled. 3.2.1 Sustainable Development Goals The United Nations have a collection of 17 numbered goals called the Sustainable De- velopment Goals (SDGs) which address global challenges such as poverty, inequality, education, and climate change [58]. The fourth goal is about quality education and is described by the United Nations as “Obtaining a quality education is the foundation to improving people’s lives and sustainable development” [57]. This project strives to create a platform where knowledge can be more conveniently exchanged worldwide. By providing professional literature, the quality of education will be improved, and the mentioned goal will be achieved. 3.3 Ethical Aspects There are some ethical implications to take into consideration while developing Lambå- Atlas. This is due to the fact that the central part of the website is a 3D model of a human body. What that body looks like with regard to skin color, gender and body type has to be considered in terms of representation. Furthermore, when performing reconstructive surgery, surgeons might sometimes have to choose between what looks “normal” and how the “ideal body” looks [29]. 3.3.1 Representation on the Main Page Human Body Ethnicities, body shapes, and genders are some examples of what distinguishes different human bodies, but since Lambå-Atlas only contains one human body, we must choose which features our 3D model should have. Even though the appearance of the body is not relevant for the practical part of the surgery [41], the ethical aspect of represen- tation is relevant. In an optimal case, the user should be able to manipulate the body and choose different features in order to represent all bodies, but this will not be imple- mented since it is not the main focus of this project. 6
3 Purpose, Aims, and Motivation Figure 2 The human body displayed on the main page of Lambå-Atlas The human body on the main page of Lambå-Atlas is a gray and muscular 3D male body as seen in Figure 2. The reason for the use of a gray skin tone is to avoid only representing one ethnicity by using an unrealistic skin colour that represents all ethnic- ities. The body has a lower than average fat percentage and even though this does not represent the average person, it facilitates the work of the surgeons when the muscle groups are easier to differentiate. In the field of flap surgery, the difference between reconstructing a male body and a female body is minimal [41]. Therefore, having two different human models, one for each sex, would be superfluous from a practical perspective. The literature of medicine tends to present the male body as the norm [37] and in order to counteract the poor representation of the females in medicine, the group intended to use a female body. However, this was not implemented since the group could not find any free 3D models of a human body that fulfilled all features of being gray, muscular, and female. 7
3 Purpose, Aims, and Motivation 3.3.2 Pictures from Previous Surgeries In addition to the 3D human body on the main page, Lambå-Atlas will contain images from old surgeries, and the representation in those images are also relevant from an ethical perspective. All images in the database of Lambå-Atlas will be uploaded by Uppsala University Hospital, and this could result in a poor demographic representation. In order to avoid poor representation, it is beneficial to have multiple users, in different parts of the world, that can upload images to the Lambå-Atlas database. This is a feature that could be added to Lambå-Atlas in the future. 3.3.3 Functionally over Appearance The purpose with flap surgery is to reconstruct parts of the human body back to “nor- mal”. Since Lambå-Atlas gives instructions about how this reconstruction should be done and how the human body should look after the surgery, it will decide what the “normal” human body looks like. These are ethical aspects that must be reflected upon. The ideal body is something that also varies over time and varies in different coun- tries [48], which also means that good representation is relevant. When a body part undergoes flap surgery, both the functionality and the appearance of the body need to be reconstructed. The importance of the appearance after the surgery differs from case to case, since maintaining functionality is a priority. Every individual user of Lambå-Atlas has different conditions, qualifications and financial preconditions. To satisfy a variety of users, Lambå-Atlas needs to give more than one solution to each surgical problem. When displaying multiple surgeries, the order in which the surgi- cal suggestions appear will also affect what Lambå-Atlas defines as normal. To create a system where this is determined by the users, rather than the creators, the surgical suggestions will be sorted according to their average ratings given by the users which describe how helpful the information has been. 3.4 Delimitations Before starting development, some delimitations had to be made. An initial idea for the 3D body was that clicking on a body part would zoom in on that body part, revealing even smaller clickable body parts. This became a delimitation due to the following reasons. The first reason is that the user would already able to zoom freely on the body, making it redundant to have an on-click zoom. The second reason is that dividing body parts into smaller components results in difficult navigation of the 3D body. The final 8
4 Related Work reason is that some small body parts would not require their own set of instructions. Another delimitation is that a female body was not implemented due to the lack of free 3D models of a female body that met the other requirements of having an unrealistic skin tone and being muscular. The current male 3D body that is implemented was not designed by the members of this project, meaning that ethical problems that were in- tended to be avoided, may exist. These implications are previously discussed in Section 3.3.1. 4 Related Work There are programs that solve similar problems to Lambå-Atlas, but do it differently or only a part of the problem this project aims to solve. Below, some related works are discussed and how their purpose, functionality, or design relates to Lambå-Atlas. 4.1 Similar Purpose and Functionality to Lambå-Atlas There are programs with similar purpose and functionality to Lambå-Atlas. There are, however, differences between the programs that set the programs apart. What these programs are and how they differ from Lambå-Atlas will be discussed in this section. 4.1.1 Microsurgeon.org A similar program to Lambå-Atlas is the website microsurgeon.org [26], whose goal is to “provide educational material relevant to microsurgery and reconstructive plastic surgery” [27]. Microsurgeon.org provides texts, and illustrations on microsurgery for educational purposes, as well as video lectures, showing how to perform microsurgery. Their filter function is implemented as a list with different types of flaps for micro- surgery, such as “head and neck” and “chest and back” [26]. In practice, this filter function is similar to the one on Lambå-Atlas, although Lambå-Atlas uses a visual filter in the form of a 3D body. While microsurgeon.org provides information about microsurgeries, the information is more theoretical than practical. This is one point where this site and Lambå-Atlas differ. The general purpose of the two programs is similar, which is to educate on microsurgery and reconstructive plastic surgery, although Lambå-Atlas strides to make an impact in developing countries. 9
5 Method 4.1.2 Touch Surgery: Surgical Videos Touch Surgery: Surgical Videos [31] is an app for learning or preparing for surgical procedures. It includes surgeries for reconstructive plastic surgeries as well as for other types of surgeries. These surgeries are presented using videos or simulations, illustrating how to perform the procedure. For each surgery, background information is given, as well as who authored the video/simulation. The different surgeries can be filtered using broad terms, such as ”Plastic, Reconstructive and Aesthetic Surgery”. This app provides similar information as Lambå-Atlas, and has a larger database of surgeries. Lambå-Atlas however, has a different filter function using a 3D-body, which is a more specific filter, making it easier to find the correct surgery. 4.2 Similar Design to Lambå-Atlas ZygoteBody [61] is a website that renders a 3D human body. The website, created by Zygote Media Group, has a similar front-end experience as Lambå-Atlas. Displaying a 3D body with different types of functionality, e.g., zooming, rotating and selecting body parts, is the main point of both ZygoteBody and Lambå-Atlas. ZygoteBody gives the ability to view the anatomy of the body in different layers, a feature that would be beneficial to Lambå-Atlas as well. The main differentiator between the two platforms being the ability to get related information from the 3D body directly on the website. 5 Method This section will present the methods used to design and create the website. Lambå- Atlas is written in React and in order to display 3D graphics the library Three.js is used. The relational database management system SQLite is used to store the surgeries and lastly the framework Django is used to create the back end and API for Lambå-Atlas. The reason these packages and frameworks were chosen and which alternatives that could have been used will be discussed. 5.1 Choice of Software Platform Accessibility is an essential part of Lambå-Atlas because a goal of this project is that it should be accessible by surgeons all around the world. Therefore, a website that is 10
5 Method accessible from mobiles phones, tablets, and desktops has been developed. The reason behind why the website needs to be mobile friendly is because, in general, the access to desktops is more limited than the access to mobile phones [53]. An alternative would be to create a mobile application for both Android and iOS devices, but since mobile appli- cations are only accessible on mobile phones, the latter was chosen. Another alternative would be to create Lambå-Atlas as a desktop application, making it for desktop users only, but this idea was eliminated since a website is accessible from a wider scheme of devices than just desktops. 5.2 Designing a Prototype in Figma The prototype for the project was created with the help of an interface design website called Figma [33]. Figma is free to use and users can design prototypes for different types of user interfaces. With the help of this website, the prototypes seen in Figure 3 and Figure 4 were designed. The reason for not choosing other design tools, such as Sketch [50] or Adobe XD [24], is mainly because of Figma’s real-time collabora- tion feature. This feature enables for different users to edit the same design and thus made it easier to edit and give feedback on the prototypes. Another reason for favoring Figma over Sketch and Adobe XD is that Figma can be used both as a website and a desktop application, whereas the others can only be used as desktop applications. This makes Figma more flexible, as it can be accessed using only a browser and an internet connection. 11
5 Method Figure 3 Figma prototype of the page where the user has chosen a body part and is shown the 3D-model with a list of surgeries Figure 4 Figma prototype of the page where the user has chosen a surgery and is shown the 3D-model with surgical instructions 12
5 Method 5.3 Front-end Programming Language The programming language for the front end had to be chosen to simplify the integration of 3D graphics on a website. With this in mind, JavaScript was chosen, due to its versa- tility and extensive list of libraries. It is used by 97.9% of all websites, where Google, Amazon, and Facebook are some of the most popular ones [46]. Other alternatives to JavaScript such as TypeScript [22] and CoffeeScript [6] are usually transcompiled, meaning that the source code is compiled by translating a programming language to JavaScript. While these languages may have their own benefits, such as improving error management or readability, they are not as widely used and well-known as JavaScript and thus offer less documentation online [40]. 5.4 JavaScript Libraries The 3D library Three.js was used for the implementation of the 3D model. Three.js uses WebGL to draw 3D. WebGL is a low-level system that only draws points, lines, and triangles, which therefore requires a lot of code to create a detailed object. Three.js simplifies this process by handling scenes, lights, materials, and other components that would have to be coded manually if WebGL was to be used directly [21]. React is another well-known library that was used for building the user interface. The library offers resources that provide a responsive user interface and reduce develop- ment time by having reusable components [45]. The combination of Three.js and React allows for the usage of other available libraries and renderers. React Three Fiber was used as a React renderer for Three.js [17]. Its task is to man- age how the code components, which form the React tree, turn into the underlying platform calls [5]. The renderer allows for a scene to be built with re-usable and self- contained components, with no limitations on Three.js regarding functionality or perfor- mance [17]. React Three Drei is an API made for React Three Fiber and the former was used to control components such as cameras and controls for customizing a scene [18]. 5.5 Handling 3D Objects Lambå-Atlas uses the file format GLB to store the 3D objects and the software program Blender to modify the 3D model. This segment discusses GLB and Blender, as well as alternatives to them. File formats that fills a similar purpose as GLB are STL and OBJ while Vectary and Wings 3D are software programs similar to Blender. 13
5 Method 5.5.1 3D-file Formats There are different ways of storing information of a 3D-object. One of the alternatives is STL, coming from stereolithogoraphy, and is one of the predominant file formats, mostly used in 3D printing [47]. STL files build the object using triangular geometry, which is the most basic way of creating meshes. STL files can be encoded with both ASCII and binary representation, where binary is more compact and more used than the ASCII counterpart. STL files cannot encode color information and lack support for different materials in the same object [47]. OBJ is also one of the dominating file formats, created for the Advanced Visualizer 3D software in the 1980s [38]. With OBJ files, the objects can use meshes built up by polygons. OBJ is widely supported and has the ability to store both color and texture, but does this using a separate MTL (Material Template Library) file. OBJ files are slow and inefficient due to a big file-size. Another drawback is that OBJ has not been updated since it was launched in the 80s [38]. The file format GLB generate one of the smallest file-formats and is a further devel- opment of GLTF (Graphics Language Transmission Format). The difference between these are that GLB is encoded in binary, while GLTF is based on JSON [23]. This makes the GLB file around 33% smaller than the GLTF option. GLB files are, with the small file size, optimized for web and mobile applications. The main reason why GLB was used is due to the small file size but while this is a strength, it is also a drawback. To achieve this small file size, the file may not have the detail or material complexity for larger, complex objects [1] [23]. 5.5.2 Modifying 3D Objects Vectary [59] is a simple to use, browser based, 3D modeling software available on most browsers. The biggest strength of Vectary is that it is browser based, that means that there is no installation of software needed. Being browser based also means that the performance is limited and that some features available in other software programs are not found in Vectary [35]. Wings 3D [60] is an open source, free to use, mesh modeler. A mesh modeler is showing the vertices making up the 3D object. Being around since 2001, Wings 3D has built up a large community of users, leading to many easy to find tutorials of how to use the software [35]. Blender [55] is, like Wings 3D, an open source, free to use, mesh modeler, but also a 3D-modeling and animation program. Blender is not only used to create 3D objects, but 14
5 Method can also be used to create animated movies. The biggest strength of Blender is how big the user base is and this was the main reason why Blender was chosen. Blenders subred- dit [4], a popular discussion forum, has more than three times the users of the second- biggest 3D modeling software. Because Blenders large list of features, the learning curve can be greater than the other alternative discussed [35]. 5.6 Building the Back End For building the back end, Django [7] is used. Django is an open-source web framework in Python [16] used to ease the process of building programs, minimizing repeating code. Using Django it is possible to design and implement the database, as well as implement CRUD (Create, Read, Update, Delete) operations using the same code base, which makes development faster, by reuse of code. Furthermore, Django comes with built-in security features [19]. Secure password management is one built-in security feature in Django [14], which makes development easier as there is one less thing to worry about when developing the back end. An alternative to Django, which is also written in Python, is the framework Flask [9]. Essentially, the things done in Django could be done using Flask. However, there are a few key advantages to choosing Django. Firstly, it has a larger community, which makes finding solutions to problems easier, as there are probably others who have had the same issue before. Secondly, it has more features built-in, such as security features and a built- in ORM (object-relational mapper) [42], which converts data from relational databases to objects [28]. Lastly, unlike Flask, Django is a full-stack framework, with a built-in admin page from which the database can be directly manipulated. Using packages, the first two of these things could be done using Flask. Even so, Django was chosen due to it having these features built-in, as well as providing an admin page, resulting in saving time while developing Lambå-Atlas. In addition to Django, the Django REST framework (DRF) [8], which is a toolkit built on top of Django, is used to build the API (Application Program Interface) [2] of the system. REST (Representational state transfer) [36] is a software architectural style that defines how data transmission should be designed. The API is the part that handles the communication between the front end and the back end. DRF is not necessarily needed to build an API using Django, but it makes it easier and faster than building it from scratch. 15
6 System Structure 5.7 Choice of Database Regarding the choice of database program, the first question is whether a relational or non-relational database is more suitable for our project. The database will mainly be used for uploading new reconstructive surgeries and querying existing surgeries based on attributes such as body part, ratings, and level of difficulty. In order to give a fast ex- perience for the users, the performance while querying surgeries will be more important than the performance while uploading new surgeries. Since relation databases perform faster when querying non-key attributes [44] such as body part, ratings and difficulty, the relational database SQLite [20] is used. Apart from SQLite, Django officially supports four other relational database programs and these are MariaDB [10], MySQL [11], Oracle Database [13], and PostgreSQL [15]. There are other databases that are supported using third party software, but to ensure compatibility, one of the officially supported databases was chosen. Since the surgeries are stored as PDFs the size of the database is small and will not exceed a few gigabytes of storage. SQLite is such a small database that it makes the system fast, simply by reducing the size of the database, and having less data to work with [49]. 6 System Structure Figure 5 An overview of the system structure. It illustrates how the different parts of the system communicate with each other. The system of Lambå-Atlas includes a back end and a front end. In the back end, the database is built using SQLite and the server is built using Django with a REST API 16
6 System Structure toolkit (Django REST framework). In the front end, React is used to build the client site, where Three.js is used to render the 3D body, and the admin site is built using Django. The client site gets data from the database through the server by making HTTP requests to a REST API, as seen in Figure 5. The server then queries the database and sends the data to the website. 6.1 Admin and Client Site The front end consists of two different views. One is the client site, which is the website that contains the 3D body. The other view is the admin site, which is a website where the admins can overview, and manipulate, the content of the client site. The two sites have separate code bases, where the client site is written using JavaScript and React while the admin site is created using Python and Django. 6.1.1 Admin Site The admin view is a pre-existing part of Django, which includes a login page and a page where the admin can manipulate the database, see Appendix A. The admin site could have been created using the same code base as the client site. However, as Django already had an admin site ready to be used, it was decided to use it as it allowed for more focus to be put on the client website. 6.2 CRUD and User Permissions To manage and access the database, CRUD (CREATE, READ, UPDATE and DELETE) operations are implemented. There are three different users in the system: users, ad- mins, and superusers. The users can READ information about operations and reviews, and can CREATE, UPDATE, and DELETE their own reviews on each operation. The admins can do everything the users can, as well as CREATE, UPDATE, and DELETE operations. They can also CREATE new users and DELETE reviews. The superuser has every access and can therefore manipulate the whole database. 6.3 Communication Between the Front End and Back End The front end communicates with the back end using the Django REST framework through HTTP (Hypertext Transfer Protocol) [39] requests. HTTP is a communication 17
7 Requirements and Evaluation Methods protocol used for transmitting data over the internet. HTTP requests are used are to implement CRUD operations. GET, POST, PUT, DELETE, and PATCH are the HTTP requests used in this system. The function of these are the same as their verbs. GET is used to read information, POST is used to create, PUT and PATCH are used to update information, and DELETE is used to delete information. In order to check a user’s permission to make HTTP requests, token authentication is used. The authentication token contains information about the user and enables them to access the website without having to log in each time they visit the website [34]. This token is included in the HTTP header, which means that it is included in every HTTP request, regardless of the type of HTTP request transmitted. 6.4 Server and Database Communication The database, SQLite, is a relational database. In this system, the database is constructed by Django using an ORM, which, in this case, means that the only interaction with the database is with Django. The ORM translates objects in Python into data that can be stored in the database. 7 Requirements and Evaluation Methods This section explains the users test that were implemented in order to test the website’s functionality and usability. Furthermore, an evaluation will be presented based on the results of the tests. 7.1 User Tests User tests were carried out where reconstructive surgeons in Uppsala University Hos- pital were asked to complete certain tasks on the website. These tasks consisted of navigating the 3D-body, clicking on a specific body part and finding the right informa- tion. The tests were performed to study the user experience and the website’s usability. After completing the tests, the surgeons were asked to fill in a survey where they could share their opinions, answer five questions, and rate the website’s functionalities on scales ranging from one to five. An evaluation was then made by studying the surveys. The test and survey are found in Appendix B. Ethical implications must be considered, since the test and survey were only done by 18
8 Implementation of the 3D Model reconstructive surgeons from Uppsala University Hospital. Therefore, the opinions that were gathered from the survey could be skewed due to a lack of diversity of test subjects. However, since there is a collaboration between reconstructive surgeons from Uppsala and other countries, they can arrange for other surgeons to take part in the same test and survey, which is further discussed in Section 10. 7.2 Evaluation of User Tests The questions in the survey were answered on a scale from one to five, where a rat- ing of at least three meant that the respective functionality is sufficient for its purpose. Therefore, if the functionalities received average ratings of at least 3, they are deemed successfully developed and thus user-friendly. The results from the survey showed that the average ratings for the functionalities was a 4.5 which is thus deemed successful. The lowest average rating was received in the question where the user was asked to rate the website’s aesthetics. The average rating for that question was a 3.7, which is also considered successful. 89% of surgeons an- swered that they would recommend the website to other reconstructive surgeons. Other opinions were also given by the surgeons, mostly revolving around the lack of surgical information and the possible ways of displaying it. Some of these opinions can be used for future work and are further discussed in Section 10. Graphs displaying the exact results of the surveys can be seen in Appendix C. 8 Implementation of the 3D Model This section provides an in-depth description of the implementation of the 3D model. Procedures such as dividing and importing the body are explained. 8.1 Dividing the 3D Body Into Body Parts An OBJ file is a standard 3D image format that can be exported or imported by several 3D image editing programs such as Blender or Tinkercad [12]. These files can be con- verted to another format known as GLB, which also stores 3D objects [1]. When a file is acquired in GLB format, it is possible to run a command which produces a JavaScript file that contains code for importing the 3D object onto a project. The command is run through npm which is a critical part of JavaScript and, among other things, a com- 19
8 Implementation of the 3D Model mand line client that allows developers to install packages of open-source code [3]. By correctly dividing the 3D body in the 3D image editing programs, the conversion to a JavaScript file will include all body parts as code, consequently allowing for simple manipulation of the body parts. 8.2 3D-modeling Software The free and open source software Blender [55] was used to create the divided 3D body. The GLB file is structured such that it contains several objects within a larger file. In Figure 6 the different objects are displayed with orange outlines inside Blender. With Blender, the bisect tool was used to divide the 3D body and make the different body parts to separate objects. Using Blender, the different body parts were also named directly in the software. 20
9 Results and Discussions Figure 6 Screenshot from Blender software displaying how the body parts were divided 9 Results and Discussions This project resulted in a website that can help reconstructive surgeons in the planning and execution of surgeries. The external supervisor of this project, Dr. Morten Kildal, intends for the final website to be an aid for reconstructive surgeons by letting them navigate a 3D human model and offer surgery instructions based on a chosen body part. A reconstructive surgeon can, during the planning and execution of surgery, rotate and zoom the 3D model, choose a body part and study the surgery instructions provided. By doing this, the project aims to assist reconstructive surgeons around the world with an intuitive platform which will simplify assessment of surgery. The collaboration that this project’s stakeholder has with Tanzania, Ethiopia and Palestine, has influenced the aim 21
10 Future Work that Lambå-Atlas has to contain and sort information based on experience and medical equipment. To realize this aim, all surgical instructions have been labelled and sorted by difficulty and medical equipment required. In order to evaluate the user experience of the website, user tests were carried out on reconstructive surgeons. After testing the website, the surgeons took part in a survey where they could rate the functionalities and express their opinions about the website. The results of the surveys revealed that the majority of the surgeons found the website to be practical and helpful. A large majority of the surgeons would also recommend the Lambå-Atlas website to other reconstructive surgeons. The exact results of the survey are discussed in Section 7 and graphs for the results can be found in Appendix C. 10 Future Work The desired functionalities for the website were initially many, but were through the course of the project reduced and put into future works. This section will further present these and explain why these were not implemented. 10.1 Local Data Storage The use of local data storage was discussed early in development. This is due to the data limitations in developing countries where Lambå-Atlas could be used. However, the focus was instead prioritized on design and functionalities. The external supervisor Dr. Morten Kildal also influenced the choice of a website by sharing his personal experience in those developing countries. Dr. Kildal explained that the surgeons in those countries had access to the internet at most times and places, thus a website would be sufficient. Another reasoning for not storing data locally is that the data consists of PDF-files that are miniscule in size. In the future, Lambå-Atlas may contain larger files or be used in areas where the internet connection is worse than what Dr. Kildal experienced. Therefore, it is worth considering storing data locally if the website was to receive more users. 10.2 Storing Patient Information A future development for Lambå-Atlas could be a feature for surgeons to create and save data from previous patients on their profiles. An idea for this would be to save 22
References a patient’s name along with a description, pictures of the surgery, and which surgical instructions that were chosen on the website. By implementing this, they could at any time revisit past surgeries to evaluate which surgical methods that were superior and use that information for future surgeries. 11 Conclusions Overall, Lambå-Atlas is a new interactive way for reconstructive surgeons to tackle the problem of finding relevant information about what surgery, or procedure to use on a patient. Furthermore, the ability for surgeons to rank procedures can assist future surgeons with choosing the appropriate procedure. The available information also gives surgical teams an easy way of comparing different operations. One of the main goals of the project was to create a platform that would promote knowl- edge exchange and education for reconstructive surgeons. By providing a platform where surgical instructions —from where knowledge and education can be gained— are accessible, this goal has been fulfilled. The purpose of making a platform accessible to countries that do not have the possibility to search through big medical databases, has been an integral part of this project. The collaboration with this project’s stakeholder, The University Hospital in Uppsala, has made it possible for Lambå-Atlas to reach countries such as Tanzania, Ethiopia, and Palestine and provide their surgeons with the knowledge platform. There are benefits of introducing a platform for knowledge exchange in order to further develop the area of reconstructive surgery. Another aspect towards the development of the area is the usage of a platform as Lambå-Atlas in countries where injuries caused by war and industry are prevalent. Because of the complexity and uniqueness of such injuries, the area of reconstructive surgery will evolve. The external supervisor of this project, Dr. Morten Kildal, has been, in the light of the current war between Russia and Ukraine, working with surgeons in Ukraine, educating them about reconstructive surgery. This is a prime example of a situation where the functionality of Lambå-Atlas could be used, and thus developing the area of reconstructive surgery. References [1] “.glb file extension,” 2020, visited on 2022-04-26. [Online]. Available: https://fileinfo.com/extension/glb 23
References [2] “Vad är ett api?” 2021, visited on 2022-05-04. [Online]. Available: https://skatteverket.se/omoss/digitalasamarbeten/utvecklingavapierochoppnadata/ kunskapochinspiration/vadarettapi.4.96cca41179bad4b1aaa4b8.html [3] “About npm,” 2022, visited on 2022-04-26. [Online]. Available: https: //www.npmjs.com/about [4] “Blenders subreddit,” 2022, visited on 2022-06-14. [Online]. Available: https://https://www.reddit.com/r/blender/ [5] “Codebase overview,” 2022, visited on 2022-04-25. [Online]. Available: https://reactjs.org/docs/codebase-overview.htmlrenderers [6] “Coffeescript,” 2022, visited on 2022-05-03. [Online]. Available: https: //coffeescript.org/ [7] “Django,” 2022, visited on 2022-04-25. [Online]. Available: https://www. djangoproject.com/ [8] “Django REST framework,” 2022, visited on 2022-04-25. [Online]. Available: https://github.com/encode/django-rest-framework [9] “Flask: web development, one drop at a time,” 2022, visited on 2022-05-10. [Online]. Available: https://flask.palletsprojects.com/en/2.1.x/ [10] “MariaDB,” 2022, visited on 2022-05-03. [Online]. Available: https://mariadb.org/ [11] “MySQL,” 2022, visited on 2022-05-03. [Online]. Available: https://www.mysql. com/ [12] “.obj file extension,” 2022, visited on 2022-04-26. [Online]. Available: https://fileinfo.com/extension/obj [13] “Oracle Database,” 2022, visited on 2022-05-03. [Online]. Available: https: //www.oracle.com/database/ [14] “Password management in Django,” 2022, visited on 2022-05-03. [Online]. Available: https://docs.djangoproject.com/en/4.0/topics/auth/passwords/ [15] “PostgreSQL,” 2022, visited on 2022-04-25. [Online]. Available: https: //www.postgresql.org [16] “Python,” 2022, visited on 2022-04-25. [Online]. Available: https://www.python. org/ 24
References [17] “React-three-fiber,” 2022, visited on 2022-04-25. [Online]. Available: https: //github.com/pmndrs/react-three-fiber#readme [18] “@react-three/drei,” 2022, visited on 2022-04-25. [Online]. Available: https: //www.npmjs.com/package/@react-three/drei [19] “Security in Django,” 2022, visited on 2022-05-03. [Online]. Available: https://docs.djangoproject.com/en/4.0/topics/security/ [20] “Sqlite,” 2022, visited on 2022-04-25. [Online]. Available: https://www.sqlite. org/index.html [21] “Threejs fundamentals,” 2022, visited on 2022-05-04. [Online]. Available: https://threejs.org/manual/#en/fundamentals [22] “Typescript,” 2022, visited on 2022-05-03. [Online]. Available: https: //www.typescriptlang.org/ [23] 3d ace, “Glb files: About the format and how to use it for your business,” 2021, visited on 2022-05-08. [Online]. Available: https://3d-ace.com/blog/glb- files-about-the-format-and-how-to-use-it-for-your-business/ [24] Adobe Inc., “Adobe XD: Design the Incredible,” 2022, visited on 2022-05-09. [Online]. Accessed: https://www.adobe.com/se/products/xd.html [25] Akademiska Sjukhuset, “Plastikkirurgmottagningen,” 2022, visited on 2022- 04-05. [Online]. Accessed: https://www.akademiska.se/for-patient-och-besokare/ hitta-pa-sjukhuset/a-till-o/plastikkirurgmottagningen/ [26] R. Buntic, D. Brooks, T. E. J. Hayakawa, H. Levinson, M. McRae, and M. McRae, “microsurgeon.org,” 2022, visited on 2022-04-04. [Online]. Accessed: https://www.microsurgeon.org/ [27] R. Buntic, “microsurgeon.org,” 2022, visited on 2022-04-04. [Online]. Accessed: https://www.microsurgeon.org/microsurgeons [28] S. Campbell, “Flask vs Django: What’s the Difference Between Flask & Django?” 05 2022, visited on 2022-05-10. [Online]. Accessed: https: //www.guru99.com/flask-vs-django.html [29] L. D. Cohn and N. E. Adler, “Female and male perceptions of ideal body shapes: Distorted views among caucasian college students,” Psychology of women quar- terly, vol. 16, no. 1, pp. 69–79, 1992. 25
References [30] Daphne Pierce-Smith RN MSN David Lickstein MD Ronald Karlin , “Overview of plastic surgery - health encyclopedia - university of rochester medical center.” [Online]. Accessed: https://www.urmc.rochester.edu/encyclopedia/content.aspx? contenttypeid=85&contentid=P01131 [31] Digital Surgery Limited, “Touch surgery: Surgical videos,” 04 2022, version 6.37.0. [Online]. Accessed: https://play.google.com/store/apps/details?id=com. touchsurgery&utm source=touch-surgery-website [32] F. Fang and K. C. Chung, “An evolutionary perspective on the history of flap reconstruction in the upper extremity,” 2014, visited on 2022-04-18. [Online]. Accessed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4160825/ [33] Figma Inc., “About figma,” 2022, visited on 2022-05-08. [Online]. Accessed: https://www.figma.com/about/ [34] Fortinet, Inc, “Authentication token,” 2022, visited on 2022-05-18. [Online]. Ac- cessed: https://www.fortinet.com/resources/cyberglossary/authentication-token [35] S. Frey, “The best free 3D modeling software for beginners in 2022,” 2022, visited on 2022-05-09. [Online]. Accessed: https://all3dp.com/1/best-free-3d-modeling- software-for-beginners/#vectary [36] L. Gupta, “REST architectural constraints,” 08 2022, visited on 2022-05-18. [Online]. Accessed: https://restfulapi.net/rest-architectural-constraints/ [37] S. Hiltner and S. Oertelt-Prigione, “Sex and gender representations of myocardial infarction in german medical books,” 2017, visited on 2022-05-04. [Online]. Accessed: https://www.liebertpub.com/doi/abs/10.1089/gg.2017.0003 [38] B. Houston, “Obj 3D file format: When should you use it?” 2019, visited on 2022-05-09. [Online]. Accessed: https://www.threekit.com/blog/obj-3d-file- format-when-should-you-use-it [39] Internetstiftelsen, “Http,” 2022, visited on 2022-05-18. [Online]. Accessed: https://internetkunskap.se/artiklar/ordlista/http/ [40] N. Kharchenko, “8 javascript alternatives for web developers to consider,” 2018, visited on 2022-05-03. [Online]. Accessed: https://codeburst.io/8-javascript- alternatives-for-web-developers-to-consider-22f8d38bdfa9 [41] M. Kildal, Personal communication, April 25th, 2022. 26
References [42] M. Makai, “Object-relational mappers (ORMs),” 2022, visited on 2022-05-10. [Online]. Accessed: https://www.fullstackpython.com/object-relational-mappers- orms.html [43] National Cancer Institute US, “Upper extremity,” 2022, visited on 2022-04- 18. [Online]. Accessed: https://www.cancer.gov/publications/dictionaries/cancer- terms/def/upper-extremity [44] Z. Parker, S. Poe, and S. V. Vrbsky, “Comparing nosql mongodb to an sql db,” ACM, p. 6, 2013. [Online]. Accessed: https://dl.acm.org/doi/abs/10.1145/ 2498328.2500047 [45] J. Patel, “10 best react libraries and frameworks for developers in 2022,” 2021, visited on 2022-05-03. [Online]. Accessed: https://www.monocubed.com/blog/ best-react-libraries/ [46] Q-Success, “Usage statistics of javascript as client-side programming language on websites,” 2022, visited on 2022-05-03. [Online]. Accessed: https: //w3techs.com/technologies/details/cp-javascript [47] D. C. Rachel Kelly, “The most common 3D file formats in 2022,” 2022, visited on 2022-05-09. [Online]. Accessed: https://all3dp.com/2/most-common-3d-file- formats-model/ [48] D. Robineau and T. de Saint Pol, “Les normes de minceur: une comparaison internationale,” vol. 504, no. 9, pp. 1–4, 2013, place: Paris Publisher: Ined Éditions. [Online]. Accessed: https://www.cairn.info/revue-population-et- societes-2013-9-page-1.htm [49] Sam Bocetta, “Comparing 3 open source databases: Postgresql, mariadb, and sqlite,” 2019, visited on 2022-05-18. [Online]. Accessed: https: //opensource.com/article/19/1/open-source-databases [50] Sketch B.V, “Home,” 2022, visited on 2022-05-09. [Online]. Accessed: https://www.sketch.com/home/ [51] J. Slick, “What is 3D modeling?” 2020, visited on 2022-05-09. [Online]. Accessed: https://www.lifewire.com/what-is-3d-modeling-2164 [52] Stanford Health Care, “Flap surgery,” 2021, visited on 2022-04-18. [Online]. Accessed: https://stanfordhealthcare.org/medical-treatments/p/plastic- surgery/procedures/flap.html#about 27
You can also read