Applying Model-Driven Engineering to Stimulate the Adoption of DevOps Processes in Small and Medium-Sized Development Organizations
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Noname manuscript No.
(will be inserted by the editor)
Applying Model-Driven Engineering to Stimulate the
Adoption of DevOps Processes in Small and Medium-Sized
Development Organizations
The Case for Microservice Architecture
Jonas Sorgalla* · Philip Wizenty · Florian Rademacher · Sabine
Sachweh · Albert Zündorf
arXiv:2107.12425v1 [cs.SE] 26 Jul 2021
Received: date / Accepted: date
Abstract the organizational and technological complexity result-
Purpose: Microservice Architecture (MSA) denotes an ing from MSA poses particular challenges for small and
increasingly popular architectural style in which busi- medium-sized organizations (SMOs). We apply Model-
ness capabilities are wrapped into autonomously de- Driven Engineering to address these challenges.
velopable and deployable software components called Results: As results of the second analysis, we identify
microservices. Microservice applications are developed the challenge areas of building and maintaining a com-
by multiple DevOps teams each owning one or more mon architectural understanding, and dealing with de-
services. In this article, we explore the state of how ployment technologies. To support DevOps teams of
DevOps teams in small and medium-sized organiza- SMOs in coping with these challenges, we present a
tions (SMOs) cope with MSA and how they can be model-driven workflow based on LEMMA—the Lan-
supported. guage Ecosystem for Modeling Microservice Architec-
Methods: We show through a secondary analysis of an ture. To implement the workflow, we extend LEMMA
exploratory interview study comprising six cases, that with the functionality to (i) generate models from API
documentation; (ii) reference remote models owned by
Attention. This is a preprint draft version.
other teams; (iii) generate deployment specifications;
J. Sorgalla* and (iv) generate a visual representation of the overall
IDiAL Institute, University of Applied Sciences and Arts Dort- architecture.
mund, Otto-Hahn-Straße 27, 44227 Dortmund, Germany
E-mail: jonas.sorgalla@fh-dortmund.de
Conclusion: We validate the model-driven workflow and
ORCiD: 0000-0002-7532-7767 our extensions to LEMMA through a case study show-
P. Wizenty
ing that the added functionality to LEMMA can bring
IDiAL Institute, University of Applied Sciences and Arts Dort- efficiency gains for DevOps teams. To develop best
mund, Otto-Hahn-Straße 27, 44227 Dortmund, Germany practices for applying our workflow to maximize effi-
E-mail: philip.wizenty@fh-dortmund.de ciency in SMOs, we plan to conduct more empirical
ORCiD: 0000-0002-3588-5174
research in the field in the future.
F. Rademacher
IDiAL Institute, University of Applied Sciences and Arts Dort- Keywords DevOps, Microservice Architecture,
mund, Otto-Hahn-Straße 27, 44227 Dortmund, Germany Development Process, Model-driven Engineering
E-mail: florian.rademacher@fh-dortmund.de
ORCiD: 0000-0003-0784-9245
S. Sachweh 1 Introduction
IDiAL Institute, University of Applied Sciences and Arts Dort-
mund, Otto-Hahn-Straße 27, 44227 Dortmund, Germany
E-mail: sabine.sachweh@fh-dortmund.de
Microservice Architecture (MSA) is a novel architec-
tural style for service-based software systems with a
A. Zündorf
Department of Computer Science and Electrical Engineering,
strong focus on loose functional, technical, and organi-
University of Kassel, Wilhelmshöher Allee 73, 34121 Kassel, Ger- zational coupling of services [53]. In a microservice ar-
many chitecture, services are tailored to distinct business ca-
E-mail: zuendorf@uni-kassel.de pabilities and executed as independent processes. The2 Sorgalla et al.
adoption of MSA is expected to increase an applica- guages and model transformations for MSA engineer-
tion’s scalability, maintainability, and reliability [53]. It ing [62] enabling sophisticated modeling support for the
is frequently employed to decompose monolithic appli- workflow.
cations for which such quality attributes are of critical The remainder of this article is organized as follows.
scale [8]. In Sect. 2, we describe in detail the microservice archi-
MSA fosters the adoption of DevOps practices, be- tecture style particularly related to the design, devel-
cause it promotes to (i) bundle microservices in self- opment, and operation stages. In addition, we explain
contained deployment units for continuous delivery; organizational aspects that result from the use of mi-
and (ii) delegate responsibility for a microservice to croservices. Section 3 illustrates LEMMA as a set of
a single team being composed of members with het- modeling languages and tools that address the MDE
erogeneous professional backgrounds [3, 52]. Conway’s of MSA. In Sect. 4, we analyze a dataset based on an
Law [14] is a determining factor in DevOps-based MSA exploratory interview study in SMOs to identify chal-
engineering. It states that the communication structure lenging areas in engineering MSA for DevOps teams
of a system reflects the structure of its development or- in SMOs. Based on these challenge areas, we present
ganization. Thus, in order to achieve loose coupling and a model-driven workflow in Sect. 6 and describe the
autonomy of microservices, it is also crucial to divide extensions of LEMMA in order to support the work-
the responsibility for microservices’ development and flow. In this regard, Subsect. 6.2 present means to de-
deployment between autonomous DevOps teams [52]. rive LEMMA models from API documentation, Sub-
As a result, MSA engineering leads to a distributed de- sect. 6.3 presents extensions to the LEMMA languages
velopment process, in which several teams create coher- in order to assemble individual microservice models,
ent services of the same software system in parallel. Subsect. 6.4 describes additions to create a visual repre-
While various larger enterprises like Netflix1 , Spo- sentation of microservice models, Subsect. 6.5 presents
tify , or Zalando3 regularly report about their success-
2
means to specify deployment infrastructure, and Sub-
ful adoption of MSA, there are only a small number sect. 6.6 elaborates on the ability to generate infrastruc-
of experience reports (e.g., [11]) about how microser- ture code. We validate our contributions to LEMMA in
vices combined with DevOps can be successfully imple- Sect. 7. Section 8 discusses the model-driven workflow
mented in small and medium-sized development organi- and LEMMA components towards the application in
zations (SMOs) with less than 100 developers involved. DevOps teams of SMOs. We present related research in
Such SMOs typically do not have sufficient resources to Sect. 9. The article ends with a conclusion and outlook
directly apply large-scale process models such as Scrum on future work in Sect. 10.
at Scale [13, 76] in terms of employees, knowledge, and
experience.
To support SMOs in bridging the gap between avail- 2 Background
able resources and required effort for a successful adop-
tion of DevOps-based MSA engineering, we (i) inves- This section provides background on the MSA approach
tigate the characteristics of small- and medium-scale and its relation towards the DevOps paradigm. It de-
microservice development processes; and (ii) propose tails special characteristics in the design, development,
means to reduce complexity and increase productivity operation, and organization of microservice architec-
in DevOps-based MSA engineering within SMOs. More tures and their realization.
precisely, the contributions of our article are threefold.
First, we identify challenges of SMOs in DevOps-based
MSA engineering by analyzing a data set of an ex-
2.1 General
ploratory qualitative study and linking it with exist-
ing empirical knowledge. Second, we employ Model- MSA is a novel approach towards the design, devel-
driven Engineering (MDE) [12] to introduce a work- opment, and operation of service-based software sys-
flow for coping with the previously identified challenges tems [53]. Therefore, MSA promotes to decompose the
in DevOps-based MSA engineering for SMOs. Third, architecture of complex software systems into services,
we present and validate extensions to LEMMA (Lan- i.e., loosely coupled software components that interact
guage Ecosystem for Modeling Microservice Architec- by means of predefined interfaces and are composable to
ture), which is a set of Eclipse-based modeling lan- synergistically realize coarse-grained business logic [24].
1 https://netflix.github.io Compared to other approaches for architecting ser-
2 https://labs.spotify.com vice-based software systems, e.g., SOA [24], MSA puts a
3 https://opensource.zalando.com strong emphasis on service-specific independence. ThisMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 3
independence distinguishes MSA from other approaches work traffic at runtime [45]. Although there exist ap-
w.r.t. the following features [53, 52, 63]: proaches such as Domain-driven Design (DDD) [25] to
support in the systematic decomposition and granular-
– Each microservice in a microservice architectures fo-
ity determination of a microservice architecture [53],
cuses on the provisioning of a single distinct capa-
their perceived complexity hampers widespread adop-
bility for functional or infrastructure purposes.
tion in practice [27, 8].
– A microservice is independent from all other archi-
An additional specific in microservice design stems
tecture components regarding its implementation,
from MSA’s omission of explicit service contracts [56].
data management, testing, deployment, and opera-
By contrast to SOA, MSA considers the API of a
tion.
microservice its implicit contract [84], thereby dele-
– A microservice is fully responsible for all aspects re-
gating concerns in API management, e.g., API ver-
lated to its interaction with other architecture com-
sioning to microservices [72]. Consequently, microser-
ponents, ranging from the determination of commu-
vices must ensure their compatibility with possible con-
nication protocols over data and format conversions
sumers and also inform them about possible interac-
to failure handling.
tion requirements. Furthermore, implicit microservice
– Exactly one team is responsible for a microservice
contracts foster ad hoc communication, which increases
and has full accountability for its services’ design,
runtime complexity and the occurrence of cyclic inter-
development, and deployment.
action relationships [77].
Starting from the above features, the adoption of
MSA may introduce increases in quality attributes [38]
such as (i) scalability, as it is possible to purposefully 2.3 Development Stage
run new instances of microservices covering strongly de-
manded functionality; (ii) maintainability, as microser- By contrast to monolithic applications, which rely on
vices are seamlessly replaceable with alternative imple- a holistic, yet vendor-dependent technology stack [18],
mentations; and (iii) reliability, as it delegates responsi- microservice architectures foster technology heterogene-
bility for robustness and resilience to microservices [53, ity [53]. Specifically, due to the increase in service-
18,19]. Additionally, MSA fosters DevOps and agile specific independence, each microservice may employ
development, because its single-team ownership calls those technologies that best fit a certain capability.
for heterogeneous team composition and microservices’ Typical technology variation points [59] comprise pro-
constrained scope fosters their evolvability [79, 16]. gramming languages, databases, communication pro-
Despite its potential for positively impacting the tocols, and data formats. However, technology het-
aforementioned features of a software architecture and erogeneity imposes a greater risk for technical debt,
its implementation, MSA also introduces complexity additional maintainability costs, and steeper learning
both to development processes and operation [16, 78, curves, particularly for new members of a microservice
72]. Consequently, practitioners in SMOs perceive the team [77].
successful adoption of MSA as complex [8]. Challenges
that must be addressed in MSA adoption are spread
across all stages in the engineering process, and thus 2.4 Operation Stage
concern the design of the architecture, its development
and operation. Furthermore, MSA imposes additional MSA usually requires a sophisticated deployment and
demands on the organization of the engineering pro- operation infrastructure consisting of, e.g., continuous
cess. delivery systems, a basic container technology and or-
chestration platform, to cope with MSA’s emphasis of
maintainability and reliability [79]. In addition, mi-
2.2 Design Stage croservices often rely on further infrastructure compo-
nents such as service discoveries, API gateways, or mon-
A frequent design challenge in MSA engineering con- itoring solutions [4], which lead to additional adminis-
cerns the decomposition of an application domain into tration and maintenance effort. Consequently, microser-
microservices that each have a suitable functional gran- vice operation involves a variety of different technical
ularity [31, 72]. Too coarse-grained microservice capa- components, thereby resulting in a significant complex-
bilities neglect the aforementioned benefits of MSA in ity increase compared to monolithic applications [72].
terms of service-specific independence. Too fine-grained Furthermore, technology heterogeneity also con-
microservices, on the other hand, may require an inef- cerns microservice operation w.r.t. technology varia-
ficiently high amount of communication and thus net- tion points like deployment and infrastructure technolo-4 Sorgalla et al.
gies [59]. Particularly the latter also involve indepen- challenges in MSA engineering (cf. Sect. 2) by means of
dent decision-making by microservice teams. For ex- Model-driven Engineering (MDE) [12].
ample, there exist infrastructure technologies, e.g., to To this end, LEMMA refers to the notion of archi-
increase performance or resilience, which directly focus tecture viewpoint [39] to support stakeholders in MSA
on a microservice [4]. Hence, teams are basically free engineering in organizing and expressing their concerns
to decide for suitable solutions based on different cri- towards a microservice architecture under development.
teria such as compatibility with existing microservice More specifically, LEMMA clusters four viewpoints on
implementations or available experience. microservice architectures. Each viewpoint targets at
least one stakeholder group in MSA engineering, and
comprises one or more stakeholder-oriented modeling
2.5 Organizational Aspects languages.
The modeling languages enable the construction of
The use of MSA requires a compatible organizational microservice architecture models and their composi-
structure, i.e., following Conway’s law, a structure that tion by means of an import mechanism. As a result,
corresponds to the communication principle of mi- LEMMA allows reasoning about coherent parts of a
croservices. This results in the necessity of using sepa- microservice architecture [39], e.g., to assess quality at-
rate teams, each of which is fully responsible for one or tributes and technical debt of microservices [61] or per-
more services (cf. Subsect. 2.1). The requirement that form DevOps-oriented code generation [60].
a team should cover the entire software lifecycle of its The following paragraphs summarize LEMMA’s ap-
microservices automatically leads to the need for cross- proach to microservice architecture model construction
functional teams. In order to ensure collaboration be- and processing.
tween teams, large companies such as Netflix or Spotify
usually use established large-scale agile process mod-
els [17], e.g., the Scaled Agile Framework (SAFe) [65], 3.1 Microservice Architecture Model Construction
Scrum at Scale [76], or the Spotify Model [70]. Estab-
lishing such a form of organization and to establish or- Figure 1 provides an overview of LEMMA’s modeling
ganizational alignment may require upfront efforts [53]. languages, their compositional dependencies and the
Thus, MSA fosters DevOps practices, which can re- addressed stakeholders in MSA engineering.
sult in lowered cost and accelerate the pace of product LEMMA’s Domain Data Modeling Language [62]
increments [52]. To this end, it is critical to foster a allows model construction in the context of the domain
collaborative culture within and across teams to pro- viewpoint on a microservice architecture. Therefore, it
mote integration and collaboration among team mem- addresses the concerns of domain experts and microser-
bers with different professional backgrounds [48]. vice developers. First, the language aims to mitigate the
A key enabler of a collaborative culture is the exten- complexity of DDD (cf. Sect. 2) by defining a minimal
sive automation of manual tasks to prevent the mani- set of modeling concepts for the construction of do-
festation of inter-team and extra-team silos [48]. Specif- main concepts, i.e., data structures and list types, and
ically, it relieves people from personal accountability the assignment of DDD patterns, e.g., Entity or Value
for a task and may thus help in reducing existing an- Object [25]. Additionally, it integrates validations to en-
imosities of team members with different professional sure the semantically correct usage of the patterns. Sec-
backgrounds [44]. ond, the language considers underspecification in DDD-
Another pillar of a collaborative culture is knowl- based domain model construction [58], thereby facilitat-
edge sharing following established formats and guide- ing model construction for domain experts. However,
lines [48]. It aims to mitigate the occurrence of insuffi- microservice developers may later resolve underspec-
cient communication, which can be an impediment in ification to enable automated model processing [60].
both MSA and DevOps [16, 64]. All other LEMMA modeling languages depend on the
Domain Data Modeling Language (cf. Fig. 1) because
it provides them with a Java-aligned type system [62]
3 Language Ecosystem for Modeling given Java’s predominance in service programming [66,
Microservice Architecture 8].
LEMMA’s Service Modeling Language [62] ad-
In our previous works we developed LEMMA [59, 62]. dresses the concerns of microservice developers
LEMMA is a set of Eclipse-based modeling languages (cf. Fig. 1) in the service viewpoint on a microservice ar-
and model transformations that aims to mitigate the chitecture. One goal of the Service Modeling LanguageMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 5
Figure 1 Overview of LEMMA’s modeling languages, their compositional dependencies and addressed stakeholders. Arrow semantics
follow those of UML for dependency specifications [54].
is to make the APIs of microservices explicit (cf. Sect. 2) vice models and assign modeled microservices to con-
but keeping their definition as concise as possible based tainers. Additionally, it is possible to express the de-
on built-in language primitives. That is, the language pendency of containers on infrastructure nodes such as
provides developers with targeted modeling concepts service discoveries or API gateways [4]. By providing
for the definition of microservices, their interfaces, op- microservice operators with a dedicated modeling lan-
erations and endpoints. LEMMA service models may guage we aim to cope with operation challenges in MSA
import LEMMA domain models to identify the respon- engineering (cf. Sect. 2). First, the Operation Modeling
sibility of a microservice for a certain portion of the Language defines a unified syntax for the modeling of
application domain [53] and type operation parameters heterogeneous operation nodes of a microservice archi-
with domain concepts. tecture. Second, it is flexibly extensible with support for
LEMMA’s Technology Modeling Language [59] con- operation technologies, e.g., for microservice monitor-
siders technology to constitute a dedicated architec- ing or security, leveraging LEMMA technology models
ture viewpoint [37] that frames the concerns of tech- (cf. Fig. 1). Third, operation models may import other
nology-savvy stakeholders in MSA engineering, i.e., mi- operation models, e.g., to compose the models of differ-
croservice developers and operators (cf. Fig. 1). The ent microservice teams to centralize specification and
Technology Modeling Language enables those stake- maintenance of shared infrastructure components such
holder groups to construct and apply technology mod- as service discoveries and API gateways.
els. A LEMMA technology model modularizes infor-
mation targeting a certain technology relevant to mi-
croservice development and operation, e.g., program- 3.2 Microservice Architecture Model Processing
ming languages, software frameworks, or deployment
technologies. Furthermore, it integrates a generic meta- LEMMA relies on the notion of intermediate model
data mechanism based on technology aspects [59]. Tech- representation [40] to facilitate the processing of con-
nology aspects may, for example, cover annotations structed models. To this end, LEMMA integrates a
of software frameworks. LEMMA service and opera- set of intermediate metamodels and intermediate model
tion models depend on LEMMA technology models transformations. The intermediate metamodels define
(cf. Fig. 1) and import them to apply the contained the concepts to which the elements of an intermediate
technology information to, e.g., modeled microservices model for a LEMMA model conform. An intermediate
and containers. In particular, LEMMA’s Technology model transformation is then responsible for the auto-
Modeling Language aims to cope with technology het- mated derivation of an intermediate model from a given
erogeneity in MSA engineering (cf. Sect. 2) by making input LEMMA model.
technology decisions explicit [73]. This approach to model processing yields several
LEMMA’s Operation Modeling Language [62] benefits. First, intermediate metamodels decouple mod-
addresses the concerns of microservice operators eling languages from model processors. Consequently,
(cf. Fig. 1) w.r.t. the operation viewpoint in MSA en- languages can evolve independently from processors as
gineering. To this end, the language integrates prim- long as the intermediate metamodels remain stable.
itives for the concise modeling of microservice con- For example, it becomes possible to introduce syntactic
tainers, infrastructure nodes, and technology-specific sugar in the form of additional shorthand notations for
configuration. To model the deployment of microser- language constructs. Second, intermediate metamodels
vices, LEMMA operation models import LEMMA ser- enable to incorporate language semantics into interme-6 Sorgalla et al.
diate models so that model processors need not antic- The first compartment shows the structure of in-
ipate them. For instance, LEMMA allows modeling of termediate model transformations with LEMMA based
default protocols for communication types within tech- on the example of a service model constructed with
nology models. In case a service model does not explic- LEMMA’s Service Modeling Language. The service
itly determine a protocol, e.g., for a microservice, the model imports a variety of domain models and tech-
default protocol of the service’s technology model ap- nology models constructed with LEMMA’s Domain
plies implicitly. The intermediate transformation, which Data Modeling Language and Technology Modeling
converts a service model into its intermediate repre- Language, respectively. As a preparatory step, the ser-
sentation, makes the default protocol explicit. Thus, vice model is transformed into its intermediate rep-
model processors can directly rely on this information resentation by means of LEMMA’s Intermediate Ser-
and need not determine the effective protocol for a mi- vice Model Transformation. Similarly, each imported
croservice themselves. domain model is transformed into a corresponding in-
Next to intermediate model representations, termediate domain model leveraging LEMMA’s Inter-
LEMMA also provides a model processing framework4 , mediate Domain Model Transformation. To this end,
which facilitates the implementation of Java-based the transformation algorithm restores the existing im-
model processors, e.g., for microservice developers port relationships between service models and domain
without a strong background in MDE. To this end, models for their derived intermediate representations.
the framework leverages the Inversion of Control However, the algorithm does not invoke intermediate
(IoC) design approach [41], and its realization based transformations on technology models imported by ser-
on the Abstract Class pattern [71] and Java anno- vice models. Instead, the applied technology informa-
tations [29]. In addition, the framework implements tion becomes part of intermediate service models so
the Phased Construction model transformation design that model processors can directly access them. There-
pattern [46]. That is, the framework consists of several fore, LEMMA treats technology models and model pro-
phases including phases for model validation and code cessors as conceptual unities, i.e., a model processor for
generation. To implement a phase as part of a model a certain technology must be aware of the semantics
processor, developers need to provide an implemen- of the elements in its technology model and be capa-
tation of a corresponding abstract framework class, ble in interpreting their application, e.g., within service
e.g., AbstractCodeGenerationModule, and augment models.
the implementation with a phase-specific annotation, The second compartment of Fig. 2 concerns model
e.g., @CodeGenerationModule. At runtime, model processing. A LEMMA model processor constitutes
processors pass control over the program flow to the an implementation conform to LEMMA’s model pro-
framework. The framework will then (i) parse all given cessing framework, which thus provides the processor
intermediate LEMMA models; (ii) transform them into with capabilities for model parsing and phase-oriented
object graphs, which abstract from a concrete model- model processing. Typical results from processing ser-
ing technology; and (iii) invoke the processor-specific vice models comprise (i) executable microservice code;
phase implementations with the object graphs. As a (ii) shareable API specifications, e.g., based on Ope-
result, the added complexities of MDE w.r.t. model nAPI5 ; (iii) event schemata, e.g., for Apache Avro6 ; and
parsing and the construction of Abstract Syntax Trees (iv) measures of static complexity and cohesion metrics
as instantiations of language metamodels [12] remain applicable to MSA [36, 2, 34, 23, 7].
opaque for model processor developers. Moreover,
LEMMA’s model processing framework provides
means to develop model processors as standalone 4 DevOps-Related Challenges in Microservice
executable Java applications. This characteristic is Architecture Engineering of SMOs
crucial for the integration of model processors into
continuous integration pipelines [42], which constitute In this section, we present an empirical analysis of mi-
a component in DevOps-based MSA engineering [5, 8]. croservice development processes (cf. Subsect. 2.5) in
Figure 2 illustrates the interplay of intermediate SMOs with the goal of identifying SMO-specific chal-
model transformations, and the implementation and ex- lenges in microservice engineering. For this purpose, we
ecution of model processors with LEMMA. perform a secondary analysis [35] of transcribed qual-
itative interviews from one of our previous works [74].
Figure 2 comprises two compartments.
While the initial analysis of the data through inductive
4 https://github.com/SeelabFhdo/lemma/tree/master/de. 5 http://spec.openapis.org/oas/v3.1.0
fhdo.lemma.model_processing 6 https://avro.apache.orgMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 7
Figure 2 Example for the interplay of LEMMA’s intermediate model transformations and model processors based on its model
processing framework.
open coding, our secondary analytical procedure specif- (Column I) with software architects whereby I4 covered
ically aims to identify challenges and obstacles during two cases.
the development process. As shown Table 1, we distinguish the cases into
greenfield (new development from scratch), templated
greenfield (new development based on legacy system),
4.1 Study Design and migration (transformation of a monolithic legacy
system into an MSA-based system) (Column Type).
The study from which the dataset emerged is a as a We further categorize each devlopment process by the
comparative multi-case study [83]. The aim of the study domain of the microservice application under develop-
was to gain exploratory insights into the development ment (Column Domain). The number of microservices
processes of SMOs. To this end, in-depth interviews present in the application at the time of the interview
were conducted on-site in 2019 with five software ar- (Column #Services), number of people (Column #Ppl)
chitects, each from a different company, and afterwards and teams (Column #Teams) involved vary depending
transcribed. The interviews were conducted in a semi- on the case. Case three is a special case; although there
structured manner and covered the areas of (i) applied are formally only two teams, sub-teams are formed de-
development process; (ii) daily routines; (iii) meeting pending on the customizations to be performed to the
formats; (iv) tools; (v) documentation; and (vi) knowl- microservice application, so that at certain points in
edge management. Participants were recruited from ex- time up to five teams work simultaneously on the ap-
isting contacts of our research group to SMOs. Further- plication. In all cases the interviewees stated to apply
more, we constrained participant selection to the pro- the Scrum framework [67] for internal team organiza-
fessional level or senior software architects, and SMOs tion. By contrast, the collaboration across teams was in
that develop microservice systems with equal or less all cases not following a particular formal methodology
than 100 people. or model (cf. Subsect. 2.5). In addition, all interviewees
reported that they strive for a DevOps culture [20] in
their SMOs. A detailed description of the cases can be
found in our previous work [74].
4.2 Dataset
As depicted in Table 1, the dataset includes transcripts 4.3 Analytical Procedure
and derived paraphrases covering six different cases
(Column C) of microservice development processes in For the analysis of the dataset, we used the Constant
SMOs. In total, we conducted five in-depth interviews Comparison method [69]. That is, we rescreened exist-8 Sorgalla et al.
Table 1 Overview of explored SMO cases [74].
C I Type Domain #Services #Ppl #Teams
Templated
C1 I1 Public Administration 60 ≈30 5
Greenfield
C2 I2 Migration B2B E-Commerce 8 10 3
C3 I3 Greenfield IoT 18 28 2 (up to 5)
C4 Migration B2B E-Commerce 34 ≈10 2
I4
C5 Migration B2C E-Commerce 8 ≈10 2
Templated
C6 I5 Logistics 15-20 75 ≈10
Greenfield
ing paraphrases and marked challenges and/or solutions and C6, the system development initially started with
that our interviewees told us about with corresponding fewer people, and as the software product became suc-
codes for challenges, obstacles, and solutions. We then cessful, more people and teams were added. Regarding
used the coded statements across all cases to combine this development and the common architectural under-
similar statements to higher-level challenges. standing I1 states that “From one agile team to multiple
agile teams is a huge leap, you have to regularly adapt
and question the organization. [...] you need a common
4.4 Study Results and Challenges
understanding of the architecture and a shared vision
of where we want to go [...], we are working on that
Our analysis of the dataset resulted in the discovery of
every day and I don’t think we’ll ever be done.”
several common challenges across all cases. Compara-
ble to other empirical studies, e.g., [80] or [33], our par- A strategy that we observed to create this common
ticipants reported about the high technical complexity architectural understanding in C1, C3, and C6 is the
and high training effort during a microservice devel- creation of new meeting formats. However, a contra-
opment process compared to a monolithic approach. dicting key aspect of the DevOps culture is to mini-
Other discovered challenges in line with existing lit- mize coordination across teams as much as possible [5].
erature, e.g., [28], concern the slicing of the business The arising problem is also experienced by our inter-
domain into individual microservices and the most suit- viewees. The more people and teams involved in ex-
able granularity of a microservice (cf. Sect. 2). changing knowledge to develop an architectural under-
In the following, we elaborate on two challenge areas standing, the more time-consuming the exchange be-
(CA) which we found to be of particular concern for comes. In the case of C6, this has led to the discon-
SMOs adopting a DevOps culture in more detail. tinuation of comprehensive knowledge exchanges due
to the excessive time involved. They now only meet
CA1: Developing, Communicating, and Stabilizing a on the cross-team level to discuss technologies, e.g.,
Common Architectural Understanding a particular authentication framework or a new pro-
gramming language. We interpret this development as
Developing a common architectural understanding of a step towards the introduction of horizontal knowl-
the architecture components of an application, espe- edge exchange formats such as Guilds in the Spotify
cially about the goals and communication relationships Model [70]. As a result, C6 is currently challenged with
of these components, in the case of MSA the microser- building a common understanding of the architecture
vices, is essential for developing a software in an organi- only through these technology-focused discussions. This
zation which follows the DevOps paradigm [5]. The in- is a problem area that is also evident in the data of other
terviewees also think that the development of a general empirical studies. For example, Bogner et al. [8] report
understanding of architecture among those involved in on the creation of numerous development guidelines by
development is an important prerequisite for granting a large development organization to enforce a common
teams autonomy and trust. architectural understanding. However, the development
For cases C2, C4 and C5 (cf. Table 1), which each of guidelines requires that architecture decisions, tech-
comprise approx. ten people and two to three teams, nology choices, and use cases are documented [32], a
the practices to achieve this understanding are Scrum practice we encountered only at C4 and C5.
Dailies [67] and regular developer meetings about the In terms of technical documentation, the teams in
current status of the architecture. However, in case of all six cases use Swagger to document the microservices’
more involved people, achieving a common understand- APIs. Other documentation, such as a wiki system or
ing is reported to be very challenging. For cases C1, C3, UML diagrams, either is not used or not kept up toMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 9
date. In almost all cases, access to the API documenta- technologies and the associated high hurdle of learning
tion is not regulated centrally, but is instead provided and integrating them into the microservice development
by the respective team through explicit requests, e.g., process. Second, the transfer of this knowledge not only
by e-mail. Only C3 has extensive and organization-wide into special units but into the individual microservice
technical documentation as it is described by I3: “Swag- teams in order to do justice to a DevOps approach.
ger is a good tool, but of course this is not completely Summarizing CA2, deployment and operation in the
sufficient, which is why we have an area where the en- SMOs studied is not in the responsibility of the teams to
tire concept of the IT platform [...] is explained. We also which the respective microservices belong. This seems
have a few tutorials.” to be due to the complexity of operation technologies
Summarizing CA1, we suspect that due to a mostly and the associated learning effort. This might particu-
volatile organization, where the number of developers larly be an issue for SMOs due to the challenging en-
and software features often grows as development pro- vironment, where there are few resources to substitute,
gresses, as well as the reported hard transition from a e.g., for a colleague who needs to learn an operation
single to multiple agile teams, SMOs are particularly technology.
affected by the challenge of implementing a common
architectural understanding as part of a successful Dev-
Ops culture. Documenting architecture decisions, deriv-
ing appropriate guidelines, and an accessible technical 5 Case Study
documentation are key factors for an efficient devel-
In this section, we present a case study that we will
opment process that become more relevant with more
use in the following sections to illustrate and validate
teams and developers involved [49] and is therefore of-
our model-driven workflow (cf. Sect. 6) to address the
ten not considered by SMOs early in the development
challenges in Sect. 4. We decided for the usage of a case
process.
study to show the applicability of our approach because
non-disclosure agreements prevent us from presenting
CA2: Complexity of Deployment Techniques and Tools our approach in the context of the explored SMO cases
(cf. Subsect. 4.2). Therefore, we selected an open source
A recurrent challenge we identified is how to deal with case study microservice architecture, which maps to the
the operation of microservice applications within the design and implementation of the explored SMO cases
development process. While cross-functional teams fol- w.r.t. the scope of our approach. More precisely, the
lowing the DevOps paradigm are mentioned in the lit- case study (i) employs Swagger for API documenta-
erature, e.g., [52], as being recommended for the im- tion (cf. Subsect. 4.4), (ii) uses synchronous and asyn-
plementation of microservices architectures, in each of chronous communication means, (iii) is mainly based
the researched cases we found specialized units for op- on the Java programming language, and (iv) the num-
erating microservices instead of operators included as a ber of software components matches the smaller SMOs
part of a microservice team. In C1, C2, and C6 we en- in our qualitative analysis (cf. Subsect. 4.2).
countered entire teams solely dedicated to operational The case study is based on a fictional insur-
aspects. In all cases, the development process included ance company called Lakeside Mutual [75]. The ap-
a handover of developed services to those specialized plication serves to exemplify different API patterns
units for operating the microservice application. Al- and DDD for MSA. The application comprises sev-
though most interviewees were aware that this contra- eral micro-frontends [57], i.e., semi-independent fron-
dicts the ownership principle of microservices (cf. Sub- tends that invoke backend functionality, and microser-
sect. 2.1) and they all stated to try to establish a pure vices centered around the insurance sector, e.g., cus-
DevOps without specialized teams, the effort to learn tomer administration, risk management, and customer
the basics of the necessary operational aspects is per- self-administration functions. The application’s source
ceived as high. In this regard I2 comments “The com- code as well as documentation is publicly available on
plexity (note: of cloud-based deployment platforms) is GitHub7 .
already very, very high, you know. I would say that each Figure 3 depicts the architectural design of the
of these functions in such a platform is a technology in Lakeside Mutual application. Overall it consists of five
itself that you have to learn.” In contrast to operations, functional backend microservice. Each microservice is
the SMOs are successful in including other professions, aligned with a micro-frontend.
such as UI/UX, as parts of their cross-functional teams.
Our data indicates that this is due to two main reasons. 7 https://github.com/Microservice-API-Patterns/
First, the inherently high complexity of the operational LakesideMutual10 Sorgalla et al.
Lakeside Mutual
«Micro-Frontend» «Micro-Frontend» «Micro-Frontend» «Micro-Frontend»
Customer Management Frontend Customer Self-Service Frontend Policy Management Frontend Risk Management Client
«http» «http» «http» «gRPC»
«Functional Microservice» «Functional Microservice» «Functional Microservice» «Functional Microservice»
Customer Management Backend Customer Self-Service Backend Policy Management Backend Risk Management Server
«amqp»
«Infrastructure Microservice»
«amqp» «amqp»
«http» Actice MQ
«http»
«http»
«http»
«http»
«Functional Microservice» «Infrastructure Microservice» «Infrastructure Microservice»
«http» «http»
Customer Core Eureka Server Spring Boot Admin
«http»
Key:
«Micro-Frontend» Micro-Frontend that provides UI components, e.g., service-specifix views. Asynchronous Communication
Micro-Frontend
«Functional Microservice» Microservice providing business functionalities via REST interfaces. Synchronous Communication
Microservice
«Infrastructure Microservice» Backend Microservice offering infrastructural functions for, e.g., service discovery or messaging.
Microservice
Figure 3 Structure of the case study Lakeside Mutual microservice application.
Except for the Risk Management Server, all mi- 6 A Model-Driven Workflow for Coping with
croservices are implemented in Java8 using the Spring DevOps-Related Challenges in Microservice
framework9 . A micro-frontend communicates with its Architecture Engineering
aligned microservice using RESTful HTTP [26]. Ad-
ditionally, the Risk Management Client and Risk This section proposes a model-driven workflow based on
Management Server communicate via gRPC. For inter- LEMMA (cf. Sect. 3) to cope with the challenges iden-
nal service to service communication, the software sys- tified in Sect. 4. More precisely, the workflow provides
tem also relies on synchronous RESTful HTTP, but also a common architectural understanding of a microser-
on asynchronous amqp messaging over an Active MQ vice application (cf. Challenge CA1 in Subsect. 4.4),
message broker. The Customer Management Backend and reduces the complexity in deploying and operating
and the Customer Core services also provide generated microservice architectures (Challenge CA2).
API documentations based on Swagger10 . In the following subsections, we present the design
of the workflow (cf. Subsect. 6.1). Next, we describe
Besides the functional microservices, the Lakeside the components, which we have added to LEMMA, to
Mutual application also uses infrastructural microser- support the workflow. These components include (i) in-
vices. The Eureka Server implements a Service Reg- teroperability bridges between OpenAPI and LEMMA
istry [63] to enable loose coupling between microservices models (CA1; cf. Subsect. 6.2); (ii) an extension to
and their different instances. For monitoring purposes, the Service Modeling Language to allow the import of
the Spring Boot Admin service provides a monitoring remote models (CA1; cf. Subsect. 6.3); (iii) a model
interface for the health status of individual services and processor to visualize microservice architectures (CA1;
the overall application. cf. 6.4); (iv) enhancing the Operation Modeling Lan-
guage (cf. Subsect. 6.5; and (v) code generators for
microservice deployment and operation (CA2; cf. Sub-
sect. 6.6).
Furthermore, we present in detail prototypical com-
ponents that we have added to the LEMMA ecosys-
8 https://www.java.com tem to support the workflow. These include deriving
9 https://spring.io models from API documentation (cf. Subsect. 6.2) and
10 https://swagger.io/ assembling microservice models into an architectureMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 11
model (cf. Subsect. 6.3) as a means to build a common Code-First Approach Here, the team first imple-
architectural understanding (CA1), and enriching mi- ments the microservice consisting of structure and
croservice models with deployment infrastructure mod- behavior. Based on the finished implementation, the
els (cf. Subsect. 6.5) as a means to more easily handle team creates an API Documentation, which can done
operational aspects for SMOs (CA2). manually or automatically with tools such as Swag-
To ensure replicability of our results we have pro- ger12 . Using the API documentation, a LEMMA do-
vided a GitHub repository11 which contains a docu- main model and a LEMMA service model are automat-
mentation how to setup LEMMA and our in this ar- ically derived (cf. Subsect. 6.2) and, if necessary, refined
ticle contributed extensions to it. It further contains by the team. In parallel, the team creates a LEMMA
all generated artifacts as well as sources and scripts operation model, since the information required for this
to rerun the generations. Finally, it contains a manu- kind of model cannot be derived from the API docu-
ally created set of LEMMA models which represent all mentation (cf. Subsect. 6.2).
Java-based microservices of the Lakeside Mutual case
study (cf. Sect. 5). Model-First Approach Alternatively, the team can de-
cide to first model the structure and operation of the
microservice using LEMMA. In the subsequent imple-
mentation activity, the structural aspects can be gen-
6.1 LEMMA-Based Workflow for Coping with DevOps
erated based on the previously constructed models and
Challenges
only the manual implementation of the behavior is nec-
essary (cf. Sect. 3).
Fig. 4 shows the conceptual elements and their relation-
Regardless of which of the two approaches was cho-
ships which underlie the design of our LEMMA-based
sen, at the end LEMMA domain, service, and operation
workflow for coping with the DevOps challenge areas
models are available that describe the Dev and Ops as-
(cf. Sect. 4).
pects of the microservice under development.
An Organization includes multiple DevOps Teams,
The operation model is then used to Generate a
each responsible for one or more Microservices
Deployment Specification for a container-based en-
(cf. Subsect. 2.5). The sum of all microservices forms
vironment which mitigates the complexity of the oper-
the Microservice Application that is developed by
ation (cf. Subsect. 6.6). The team refines this specifi-
the organization. Associated with a microservice is
cation as needed and then deploys the microservice. In
a corresponding documentation of its interfaces (API
parallel, the models generated during the workflow are
Documentation). For each microservice owned by it,
sent to a central model repository and made available
the team constructs a Set of LEMMA Views as a model
to the entire organization where they can be used by
representation (cf. Sect. 3). The sum of all LEMMA
other teams to gain insight and a common understand-
models forms an Architecture Model which describes
ing of the application’s architecture, e.g., by visualizing
the system’s architecture. This model can be used by
its structure.
the organization, e.g., to gain insight into existing de-
Based on the use of model transformations and code
pendencies between the microservices involved.
generation steps, in the code-first as well as the model-
Based on the conceptual elements and their rela-
first approach, we argue that the application of the
tionships, Fig. 5 shows our model-driven workflow for
workflow is possible with almost the same resources
DevOps-based microservices development in SMOs as
as the current development processes in the individ-
a UML acitivity diagram [54].
ual DevOps teams that we were able to explore as
We depict the workflow from the perspective of a cases in the empirical study (cf. Sect. 4). At its core,
single DevOps team including all steps required for the the code-first approach relies on the same development
development of a new microservice. When incremental steps, i.e., implementing structure and behavior of a
changes are made to individual aspects of a microser- microservice, as non-model-based processes in the in-
vice, only the steps affected by the changes need to be dividual teams, so that even teams without experience
performed. in MDE can adapt the flow in a non-invasive way. Be-
The process starts with the planning of the de- sides the actual implementation, the workflow provides
velopment. The team decides whether to follow a a service’s description in the form of LEMMA viewpoint
code-first or model-first approach. We support models, which can be used as a communication basis
both variants to allow the teams autonomy according and for knowledge transfer in order to create a com-
to the DevOps paradigm [5]. mon architecture understanding (CA1; cf. Subsect. 4.4)
11 https://github.com/SeelabFhdo/SN2021 12 https://swagger.io/12 Sorgalla et al.
Figure 4 Overview of the concepts within the workflow and their interrelationships represented as a UML class diagram [54].
in the organization. This can be used to, e.g., acceler- "paths":{ 1
ate verbal coordination processes between teams, im- "/cities/{postalCode}":{ 2
"get":{ 3
prove the documentation, or identify microservice bad "tags":["city-reference-data-holder"], 4
"summary":"Get the cities for a postal code.", 5
smells ([77]). In addition, by using LEMMA operation "operationId":"getCitiesForPostalCodeUsingGET", 6
models and generating deployment specifications, it is "produces":["*/*"], 7
"parameters":[{ 8
easier for teams of an SMO to address the ops aspects "name":"postalCode", 9
themselves without passing on the responsibility for de- "in":"path", 10
"description":"the postal code", 11
ployment to another unit (CA2; cf. Subsect. 4.4). This "required":true, 12
enables teams to foster the ownership principle of MSA "type":"string" 13
}], 14
(cf. Subsect. 2.1). "responses":{ 15
"200":{ 16
"description":"OK", 17
"schema":{"$ref":"#/definitions/ 18
CitiesResponseDto"}
6.2 Derivation of Microservice Models from API }, 19
Documentations ... 20
Listing 1 Excerpt of the OAS conform OpenAPICustomerCore-
To enable the model-driven workflow with sophisticated BackendService.json describing the CustomerCore microservice.
modeling support by LEMMA, we extended the ecosys-
tem with the ability to derive data and service models
from API documentation, that conforms to the Ope-
nAPI Specification13 (OAS) [55], into LEMMA mod- information from the API Model is transformed into a
eling files. OAS defines a standardized interface to de- Domain Data Model, Service Model, and a Technol-
scribe RESTful APIs. One of the most popular tools ogy Model which each correspond to their respective
implementing OAS is Swagger, which was used by all LEMMA metamodel (cf. Sect. 3). Table 2 describes
SMOs in the qualitative study (cf. Sect. 4). which OAS objects15 are mapped to which LEMMA
The transformation of OAS files into LEMMA files model kind.
can be classified as an interoperability issue in which To be able to transform the in-memory LEMMA
OAS models are to be converted into LEMMA models. models as files, we extended LEMMA with extrac-
We therefore applied the interoperability bridge process tors [9] for technology, service, and data models.
proposed by Brambilla et al. [9]. Fig. 6 shows the ap- Listing 1 and Listing 2 illustrate the application of
plied interoperability bridge process. the process.
First, using the Swagger parsing framework14 , an Listing 1 shows an excerpt of the API documen-
OAS conform API model in the YAML [6] or JSON [21] tation file of the Customer Core microservice from the
format is converted into an in-memory API Model. We case study (cf. Sect. 5). In detail, the listing presents the
then perform three model transformations where the OAS description for a HTTP GET request on the path
cities/{postCode} (Lines 2 and 3). This includes,
13 Version 3.0.3
14 https://github.com/swagger-api/swagger-parser 15 https://spec.openapis.org/oas/v3.0.3#openapi-objectMDE to Stimulate DevOps Processes in the Context of Microservice Architecture 13
Figure 5 Proposed workflow for DevOps teams for model-driven microservices development represented as a UML activity dia-
gram [54].
e.g., the unique id getCitiesForPostalCodeUsingGET the technology model OpenApi.technology (Line 3),
(Line 6) of the operation, the incoming parameters which contains, e.g., the OpenAPI-specific primitive
(Line 8 to 14), and the information that a response data types and the media types used in the Customer-
returns an object based on the CitiesResponseDto Core OAS model. Line 4 enables the OpenApi technol-
schema (Lines 15 to 19). The excerpt shows only the ogy for the com.lakesidemutual.customercore.Cus-
response for HTTP status code 200 (Line 16). OAS also tomerCore microservice whose definition starts in Lines
offers the possibility to define responses for other status 6 and 7. The microservice comprises an interface named
codes, e.g. HTTP status code 404, but these are cur- cityReferenceDataHolder which was derived by the
rently not considered in the transformation to LEMMA associated tags in the OAS model (Line 8). The inter-
in our prototypical implementation. faces consists of the operation getCitiesForPostal-
CodeUsingGET named after the OAS operationId
Listing 2 shows the LEMMA service model auto-
(Lines 18 to 22). The operation commentary section
matically transformed from the CustomerCore OAS
(Lines 9 to 14) is populated using the summary informa-
model in Listing 1. First, the results of the other
tion from OAS. The OAS path is added as an endpoint
transformations are imported into the service model.
(Line 15) and the operation classified as an HTTP GET
This includes the previously transformed LEMMA do-
request (Line 17). The OAS response associated with
main data model customerCore.data resulting from
the HTTP status code 200 is modeled as an OUT pa-
the OAS schemas (Lines 1 and 2), which contains
rameter and named returnValue (Lines 20 and 21).
all data structures such as CitiesResponseDto, and14 Sorgalla et al.
Figure 6 Process to transform OAS conform API models into LEMMA models.
Table 2 Applied mappings between OAS and LEMMA concepts.
OAS Objects LEMMA Model Description of Mapping
Info Domain Data Name and version of the Domain and its Context
Schemas Domain Data DataStructures of the Context
Inline schemas are transformed to additional DataStructures and arrays are mapped
Paths Domain Data
to ListTypes
Info Service Name and commentary of the Microservice
Tag Service Used to derive a service’s Interfaces which cluster Operations
Operations of an Interface comprising its Endpoints, Parameters, HTTP request
Paths Service
method as Aspect, and Commentary
All mentioned media types in a OAS model are transformed to DataFormats of the
Media Types Technology
RESTful HTTP Protocol.
Data Types Technology Mapped to TechnologySpecificPrimitiveTypes.
import datatypes from "customerCore.data" 1 pend on other services. In the case study (cf. Sect. 5),
as CustomerCoreAPIData 2 such a relationship is found between the microservices
import technology from "OpenApi.technology" as OpenApi 3
4 Customer Management Backend and Customer Core.
@technology(OpenApi) 5
public functional microservice 6
Such dependencies cannot be derived from an API doc-
com.lakesidemutual.customercore.CustomerCore { 7 umentation, since its purpose is to describe the provided
interface cityReferenceDataHolder { 8
--- 9 interface of a service and not the invocation of function-
**Type** GET Operation for path /cities/{postalCode} 10ality provided by other architecture components. How-
**Summary** Get the cities for a postal code. 11
**Description** 12ever, these dependencies are essential in order to be able
@required returnValue [INSERT PARAMETER DESC HERE] 13to assemble and assess an architecture model and to
--- 14
@endpoints(OpenApi::_protocols.rest: 15raise a common architectural understanding across the
"/cities/{postalCode}";) 16
@OpenApi::_aspects.GetMapping 17
whole organization (cf. Subsect. 4.4). Therefore, within
getCitiesForPostalCodeUsingGET( 18the workflow (cf. Subsect. 6.1), the dependencies should
sync in postalcode : string, 19
sync out returnValue : CustomerCoreAPIData::v100. 20be added manually by the teams in the LEMMA mod-
CustomerCoreAPI.CitiesResponseDto); 21els. This can be done during the Model Services ac-
} 22
... 23tivity when using the model-first approach and during
} 24the Refine Generated Models activity when using the
Listing 2 Excerpt of customerCore.services resulting from code-first approach.
transforming the CustomerCore API documentation.
However, LEMMA service models originally were
only able to depend on other LEMMA service models if
they are accessible in the local file system. Therefore, to
6.3 Assembling a Common Architecture Model from allow teams the expression of interaction dependencies
Distributed Microservice Models with the microservices of other teams, we have extended
LEMMA to allow external service imports. Listing 3
Microservices need to interact with each other to re- shows the service model of the Customer Management
alize coarse-grained functionality [53]. Thus, can de- Backend microservice from the case study. The mi-You can also read
