CSC301 WINTER 2022 ILIR DEMA FEB 1-2, 2022 - NOSQL GRAPH DBS. SOFTWARE ARCHITECTURE. PLANNING ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CSC301 W INTER 2022
W EEK 4 - N O SQL GRAPH DB S . S OFTWARE A RCHITECTURE .
P LANNING AND P RIORITIZING .
Ilir Dema
University of Toronto
Feb 1-2, 2022
W HAT IS N O SQL?
I NTRO TO N O SQL
• The growth of Web raised the need for larger, more
scalable storage solutions.
• a variety of key-value storage solutions were designed for
better availability, simple querying, and horizontal scaling.
• This new kind of data store became more and more robust,
offering many of the features of the relational databases.
• Different storage design patterns emerged, including
key-value storage, column storage, object storage, and the
most popular one, document storage.
R ELATIONAL VS D OCUMENT- ORIENTED DB
• In a common relational database, data is stored in different
tables, often connected using a primary to foreign key
relation.
• A program will later reconstruct the model using various
SQL statements to arrange the data in some kind of
hierarchical object representation.
• Document-oriented databases handle data differently.
• Instead of using tables, they store hierarchical documents
in standard formats, such as JSON and XML.
R ELATIONAL DB EXAMPLE blog post model - data stored in different tables:
D OCUMENT- ORIENTED DB EXAMPLE
• In a document-based database, the blog post will be stored
completely as a single document that can later be queried.
• For instance, in a database that stores documents in a
JSON format, the blog post document would probably look
like the following code snippet:
This model will allow faster read operations since your
application won’t have to rebuild the objects with every read.
W HAT IS N EO 4 J ?
A GRAPH DATABASE
• Uses graphs to store and process the data
• Data is organized into nodes and relationships
• Properties are stored in either nodes or relationships
• Recently, Neo4j and Google Cloud have teamed up to
deliver Neo4j for Google Cloud, the Neo4j graph database
delivered as a Google Cloud Platform (GCP) native
service.
• Neo4j’s native data manipulation language is Cypher.
G RAPHS
H OW ABOUT DATA ?
H OW ABOUT DATA ?
C OMPARE TO SQL
T RANSITION TO GRPAHS
M ODELING WITH GRAPHS
N EO 4 J IS FULLY ACID COMPLIANT ...
ACID
• ATOMIC: The whole transaction or nothing
• CONSISTENT: Upon completion of a transaction, the db is
structurally sound
• ISOLATION: Transactions appear to apply in isolation from
one another
• DURABLE: Once a transaction is complete, it persists,
even in case of various failuresW ELCOME TO C YPHER
N OT JUST A QUERY LANGUAGE
• Declarative, readable, expressive
• Made for CRUD on graphs
• Based on patterns
• Interacts safely with the remote database using a binary
protocol called BoltP ROPERTY G RAPHS
A PROPERTY GRAPH HAS
• Nodes (:PERSON)
• have properties ({name: ”Donald”})
• Relationships [:WORKS_WITH]
• also have properties ({company: ”Bluecat”})
A N EXAMPLE OF CREATE
CREATE
(: PERSON {name:"Donald"})
-[:WORKS_WITH {company: "Bluecat"}]->
(: PERSON {name: "Jasvir"})C YPHER WORKS BASED ON PATTERNS
W HO WORKS WITH JASVIR AT B LUECAT ?
MATCH
(p1: PERSON)
-[:WORKS_WITH {company:"Bluecat"}]->
(:PERSON {name:"Jasvir"})
RETURN
p1S OFTWARE D ESIGN T ONY H OARE : There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.
L EVELS OF DESIGN
• Architectural design (also: high-level design)
• architecture - the overall structure: main modules and their
connections
• design that covers the main use-cases of the system
• addresses the main non-functional requirements (e.g.,
throughput, reliability)
• hard to change
• Detailed design (also: low-level design)
• the inner structure of the main modules
• may take the target programming language into account
• detailed enough to be implemented in the programming
languageS OFTWARE A RCHITECTURE D EFINITION A software architecture is a description of the subsystems and components of a software system and the relationships between them. Subsystems and components are typically specified in different views to show the relevant functional and nonfunctional properties of a software system. The software architecture of a system is an artifact. It is the result of the software development activity. Buschmann et al., Pattern-Oriented Software Architecture, A System of Patterns
T HE MVC PATTERN
• Model:
• Contains Data Objects
• Encapsulates the application state
• Responds to state queries/updates
• Exposes application functionality
• View:
• Renders the model (i.e. the screen representation of the
data).
• Note: The user is not necessarily a human. For example,
programs want to view the data using some text format (e.g.
XML, or JSON)
• Sends user input to the Controller
• Controller:
• Defines application behavior
• Maps user actions to Model updates
• Controls the flow of the application.
• Defines the way the user interface reacts to the user input.T HE MVC AS A RCHITECTURAL PATTERN
MVC INTEGRATES A FEW D ESIGN PATTERNS
• Model uses Observer to keep views and controllers
updated on the latest state changes
• View and Controller implement Strategy pattern. Controller
is the behavior of the View and can be easily exchanged
with another controller if you want different behaviour.
• View uses Composite pattern to manage the components
of the display.T HE 3- TIERED ARCHITECTURE
W HAT DOES THREE TIERED MEAN ?
• The presentation tier is the front end layer in the 3-tier
system and consists of the user interface.
• The application tier contains the functional business logic
which drives an application’s core capabilities.
• The data tier comprises of the database/data storage
system and data access layer.T HE 3- TIERED ARCHITECTURE
W HERE DOES IT DIFFER FROM MVC?
• MVC and 3-tier architecture are topologically different.
• Conceptually the three-tier architecture is linear. MVC
architecture is triangular: the view sends updates to the
controller, the controller updates the model, and the view
gets updated directly from the model.
• A fundamental rule in a three tier architecture is the client
tier never communicates directly with the data tier.M ICROSERVICES ARCHITECTURE
W HAT DOES MICROSERVICES MEAN ?
• Develop a single application as a suite of small services
• Each running separately and communicating via HTTP.
• These services are independently and automatically
deployable.
• They may use different programming languages and use
different data storage technologies.R ELEASE P LANNING
• During the release planning meeting the following things
are established:
• Major release goals
• Release plan
• Potential sprint goals
• Completion date
• As each sprint progresses the burndown of story points
measure the velocity of work, which can be used to
determine progress and adapt the plan as we goS PRINT P LANNING
• The team decides (reviews) velocity - how many story
points will they do in this sprint.
• Most priority stories from the product backlog are selected,
filling up the velocity.
• team never overcommits!
• The tasks from each selected story is broken down to build
the sprint backlog
• Meeting may include additional domain experts (not part of
the team) to help answer any questions and aid in time
estimations.
• Implementation details are discussed
• Product owner must be present to answer any questions
related to the designTASK P LANNING
• Tasks are estimated in hours
• Estimation is an ideal time (without interruptions / problems)
• After all tasks have been estimated the hours are totaled
up and compared against the remaining hours in the sprint
backlog
• If there is room, the team picks more stuff from product
backlog and updates the velocity.
• All planning decisions are recorded on the tracker.T RACKING P ROGRESS
• Information about progress, impediments and sprint
backlog of tasks needs to be readily available
• How close a team is to achieving their goals is also
important
• Scrum employs a number of practices for tracking this
information:
• Task cards
• Burndown charts
• Task boards
• War roomsB URNDOWN C HART Source: Wikipedia
TASKBOARD Source: https://manifesto.co.uk/agile-concepts-scrum-task-bo
DAILY S CRUM M EETINGS
• 15 minute meeting that everyone must attend
• No sitting down, team stands in a circle and answers the
following questions: What have I done since the last
meeting?
• What am I going to accomplish between now and the next
meeting?
• What are the problems or impediments that are slowing me
down?
• It is NOT for solving problems - the Scrum Master must
ensure that all side conversations are kept to a minimum
• Solving problems happens throughout the rest of the day
• Can be evolved to meet a specific team’s requirements, but
the purpose must remain the same (status, commitment,
improvement)S PRINT R EVIEWS
• Occur on the last day of the sprint
• Team and stakeholders come together to play the game
and discuss the work accomplished
• Product owner accepts or declines the results of the sprint
• If a feature is declined, the owner will decide if it is returned
to the backlog or simply dropped
• Honesty is crucial
• Cannot discourage criticism simply because a lot of work
was put inW HAT I S D EPENDENCY I NJECTION ? D EFINITION The definition of dependency injection was first given by Martin Fowler in the blog post Inversion of Control Containers and the Dependency Injection pattern. The dependency injection is an Enterprise Design Pattern, which aim is to separate the responsibility of resolving object dependency from its behaviour. N OTE : An Enterprise Design Pattern is a Design Pattern used in enterprise applications.
F OWLER ’ S EXAMPLE
E XERCISE
• Try to write (on paper) a constructor for MovieLister.
• Identify the broken design principles ...T HE M O V I E L I S T E R C ONSTRUCTOR
public class MovieLister {
private MovieFinder finder;
public MovieLister() {
// This statement tightly coupled (1)
// the two classes because the class has a
// direct reference to a particular
// implementation of MovieFinder interface (2)
this.finder = new MovieFinderImpl();
}
// ...H OW DID WE GET IN TROUBLE ?
• Clearly, in the previous slide we used a long venerated
design principle:
• Favor composition over inheritance
• However letting a class to explicitly create an instance of
another class tightly coupled the two implementations,
increasing the dependency between them.
F RED B ROOKS : N O S ILVER B ULLET
There is no single development, in either technology or
management technique, which by itself promises even one
order-of-magnitude improvement within a decade in
productivity, in reliability, in simplicity.L ET ’ S LOOK FOR HELP !
• Well we know a design pattern used to make objects.
• The factory pattern!
• Its intent is:
• creates objects without exposing the instantiation logic to
the client.
• refers to the newly created object through a common
interfaceUML FOR THE FACTORY PATTERN
L ET ’ S APPLY IT... Notice the dependency graph is directed acyclic (DAG).
H OUSTON , WE HAVE A PROBLEM !
• Now MovieLister has a dependency with the
implementation of MovieFinderFactory.
• Notice factory is implemented as a Singleton ...
• Moreover, we certainly have more than a single
dependency in a class, which leads to a plethora of
factories rising.
• And dependencies can have dependencies, and so on.
• We get a chain of growing dependencies.
• There are two possibilities:
• The chain blows up in finite time.
• The chain stabilizes (i.e. the nightmare comes to a limit).M ARTIN F OWLER ’ S SOLUTION Take the factory idea to the limit and we create a module that is responsible to resolve dependency among classes.
D EPENDENCY I NJECTOR
• The Injector module is called dependency injector and it is
to all effects a container.
• Applying the Inversion of Control pattern, the injector owns
the life cycle of all objects defined under its scope.
• The only things we still miss are the following:
• A way to signal to the injector that a class has a certain
dependency
• A way to instruct (or configure) the injector to resolve
dependenciesI NTERLUDE : JAVA B EANS
• Since the definition of the Java programming language, the
aim of the Sun Microsystem was standardization.
• The Java Bean Specification stated that a java bean must
have:
• a default constructor
• a couple of getter and setter methods for each attribute it
owns.
• Having defined a standard way of creating objects, the Sun
built a plethora of standards and interfaces on it, that are
the basis of the JEE specification.C ONSTRUCTOR INJECTION VERSUS S ETTER INJECTION
• Since dependencies are nothing more then objects
attributes (more or less), the injector has to be instructed to
know how to fulfill these attributes.
• In Java there are three ways to set attributes of an object,
which are:
• Using the proper constructor
• Using setters after the object was build using the default
constructor
• Using Reflection mechanisms
• Once you have selected your preferred way, you will
annotate the corresponding statement with the @Inject
annotation.
• The annotation and its behaviour are defined inside a JSR.
Dependency injection is so important in the Java
ecosystem that there are two dedicated Java Specification
Requests (JSRs), i.e. JSR-330 and JSR-299.C ONSTRUCTOR DEPENDENCY INJECTION
• If you want the injector to use the constructor to inject the
dependency of a class, annotate the constructor with
@Inject.
• Every time you will ask the injector an instance of a
MovieLister, it will know that it has to use the
constructor annotated with the @Inject annotation to
build the object.
• So, the dependency are identified as the annotated
constructor parameters.
public class MovieLister {
private MovieFinder finder;
@Inject
public MovieLister(MovieFinder finder) {
this.finder = finder;
}S ETTER DEPENDENCY INJECTION
• To instruct the injector to use setter methods to create an
object, annotate the setter methods with the @Inject
annotation.
public class MovieLister {
private MovieFinder finder;
// The injector first uses the default
// constructor to build an empty object
public MovieLister() {}
// Then, the injector uses annotated setter
// methods to resolve dependencies
@Inject
public void setFinder(MovieFinder finder) {
this.finder = finder;
}
}T HE INJECTOR
• There are many implementations in the Java ecosystem of
dependency injectors. Each implementation differs from
each others essentially for these features:
• How the injector is configured to find the beans it has to
manage
• How it resolves the dependencies DAG
• How it maintains the instances of managed beans.T HE INJECTOR
• Some injectors used in practice are the following:
• Google Guice: Guice is a lightweight dependency injection
framework that uses Java configuration, implements both
types of injection (constructor and setter injection) and
maintains managed instances with a
MapW HAT IS DAGGER -2?
• Dagger-2 is a fast and lightweight dependency injection
framework.
• It is implemented through an external component which
provides instances of objects (or dependencies) needed by
other objects.
• In particular, the injection happens at run-time or at
compile-time.
• Run-time DI is usually based on reflection which is simpler
to use but slower at run-time. An example of a run-time DI
framework is Spring.
• Compile-time DI, on the other hand, is based on code
generation. This means that all the heavy-weight
operations are performed during compilation. Compile-time
DI adds complexity but generally performs faster.
• Dagger 2 falls into this category.E XAMPLE
In order to use Dagger in a project, we’ll need to add the
dagger dependency to our pom.xml:
M AVEN C ONFIGURATION
com.google.dagger
dagger
2.16
Note: Eclipse needs be configured as follows:
• Install m2e-apt
• Window -> Preferences -> Maven -> Annotation
Processing: Select "Automatically configure JDT APT"E XAMPLE
Also need to include the Dagger compiler used to convert our
annotated classes into the code used for the injections:
M AVEN C ONFIGURATION
org.apache.maven.plugins
maven-compiler-plugin
3.6.1
com.google.dagger
dagger-compiler
2.16T HE C AR EXAMPLE
• Dagger uses the standard JSR-330 annotations in many
places, one being @Inject.
• Since Dagger doesn’t support injection on private fields,
we’ll go for constructor injection.
B UILD A CAR BY INJECTING ITS COMPONENTS
public class Car {
private Engine engine;
private Brand brand;
@Inject
public Car(Engine engine, Brand brand) {
this.engine = engine;
this.brand = brand;
}
// getters and setters
}C ODE N EEDED T O P ERFORM THE I NJECTION
• Next, we’ll implement the code to perform the injection.
More specifically, we’ll create:
• a module, which is a class that provides or builds the
objects’ dependencies, and
• a component, which is an interface used to generate the
injector
• Complex projects may contain multiple modules and
components but since we’re dealing with a very basic
program, one of each is enough.C REATING A M ODULE
A Module class is annotaed with the @Module annotation,
indicating that it can make dependencies available to the
container. Then, we need to add the @Provides annotation on
methods that construct our dependencies:
A M ODULE EXAMPLE
@Module
public class VehiclesModule {
@Provides
public Engine provideEngine() {
return new Engine();
}
@Provides
@Singleton
public Brand provideBrand() {
return new Brand("CSCC01");
}
}C OMPONENTS
• Components are essentially the glue that holds everything
together.
• They are a way of telling Dagger what dependencies
should be bundled together and made available to a given
instance so they can be used.
• They provide a way for a class to request dependencies
being injected through their @Inject annotation.C OMPONENT EXAMPLE
Moving on, we’re going to create our component interface. This
is the class that will generate Car instances, injecting
dependencies provided by VehiclesModule.
Simply put, we need a method signature that returns a Car and
we need to mark the class with the @Component annotation.
Notice how we pas our module class as an argument to the
@Component annotation. If we didn’t do that, Dagger wouldn’t
know how to build the car’s dependencies.
A COMPONENT EXAMPLE
@Singleton
@Component(modules = VehiclesModule.class)
public interface VehiclesComponent {
Car buildCar();
}You can also read
