Configuration and Deployment Guide For OpenStack* Swift* Object Storage on Intel Atom Processor and Intel Xeon Processor Microservers
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Software Configuration
and Deployment Guide
Configuration and Deployment Guide
For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and
Intel® Xeon® Processor Microservers
About this Guide
This Configuration and Deployment Guide explores designing and building an object
storage environment based on OpenStack* Swift.* This guide focuses on object storage in
cloud environments, where responsiveness and cost are critical factors. High-performance,
energy-efficient microservers, such as those based on the latest generation of Intel®
Atom™ processors and Intel® Xeon® E3 processors, meet these requirements. The guide
uses data from recent benchmarks conducted by Intel® Software and Services Group on
Intel Atom processor and Intel Xeon E3 processor-based microservers.
Introduction
OpenStack Swift Object Storage (swift.openstack.org) enables a scalable, redundant
storage filesystem for cloud infrastructures, accessible using web protocols at incredible
speeds with very high availability. Swift Object Storage started as a project at the cloud
service provider, RackSpace;* it was released to the OpenStack (www.openstack.org)
community. It is available under the Apache* 2 license.
In Swift, objects are written to multiple disk drives spread across a cluster of storage
servers in the data center. But systems do not mount Swift storage like traditional
storage area network (SAN) or network attached storage (NAS) volumes. Instead,
applications use a representational state transfer (REST) application programming
interface (API) to request data stored on the Swift cluster.
The Swift Object Storage cluster scales in a near-linear fashion by adding new servers.
The software ensures data replication and integrity throughout the cluster. Should a
storage server or hard drive fail, Swift Object Storage software replicates content from
other active servers to new locations in the cluster. Because the software ensures data
replication and distribution across different storage devices, data center architects can
use industry-standard hard disk drives (HDDs), solid-state disk drives (SSDs), and servers,
including microservers running on Intel Xeon E3 processors or Intel Atom processors.
Other key characteristics of Swift Object Storage include:
• All objects stored in Swift have a URL.
•A
ll objects stored are replicated three times in unique-as-possible zones,
which can be defined as a group of drives, a node, a rack, etc.
• Each object has its own metadata.
•D
evelopers interact with the object storage system through a REST HTTP API.
•O
bject data can be located anywhere in the cluster.Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Table of Contents • The cluster scales by adding nodes, A New Look at Storage
without sacrificing performance, When considering different scalable
Introduction. . . . . . . . . . . . . . . . . . . . . . 1
which allows a more cost-effective storage solutions, several things are
Driving Forces and linear storage expansion versus fork- important to keep in mind.
Strategic Considerations. . . . . . . . . . . 2
lift upgrades.
A New Look at Storage. . . . . . . . . . . . . 2 • Scalability
• Data doesn’t have to be migrated to an
Overview of the entirely new storage system. • Durability
Swift Architecture . . . . . . . . . . . . . . . . 3 • Cost
• New nodes can be added to the cluster
Primary Objective: High Availability
without downtime. • Choice of components
and Fast Access to a Wide Variety
of Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 • Failed nodes and disks can be swapped • Compatibility and familiarity
Theory of Operation. . . . . . . . . . . . . . . 4 out with no downtime. • Cloud or in-house
Configuration and Deployment. . . . . 5 • Swift runs on cost-effective
Scalability
industry-standard hardware.
Swift Topology . . . . . . . . . . . . . . . . . . . . 5 To easily grow with expanding content
Access Tier. . . . . . . . . . . . . . . . . . . . . . . . 5 Driving Forces and users, storage systems need to
Storage Nodes. . . . . . . . . . . . . . . . . . . . . 6 and Strategic Considerations handle web-scale workloads with many
With rapidly-growing mobile user-bases concurrent readers from and writers to
Performance Considerations . . . . . . . 6
for social media and a variety of other a data store, writing and reading a wide
Storage Server Benchmarking variety of data types. Some data is
for Swift . . . . . . . . . . . . . . . . . . . . . . . . . 7 cloud services, a wide variety of struc-
tured and unstructured data needs to frequently written and retrieved, such as
COSBench Platform. . . . . . . . . . . . . . . . 7 database files and virtual machine images.
be instantly accessible, secure and
Configurations. . . . . . . . . . . . . . . . . . . . . 8 redundant, possibly stored forever, and Other data, such as documents, images,
System Under Tests (SUT) available through a variety of devices for and backups, are generally written once
Configurations. . . . . . . . . . . . . . . . . . . . . 9 a variety of applications. Storage silos and rarely accessed. Unlike traditional
utilizing protocols that are tied to specific storage file systems, Swift Object Storage
Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
applications no longer meet the needs of is ideal for storing and serving content to
Tuning and Optimization. . . . . . . . . . 10 many, many concurrent users. It is a
web-based applications. Social media,
General Service Tuning. . . . . . . . . . . . 10 online video, user-uploaded content, multi-tenant and durable system, with no
Filesystem Considerations. . . . . . . . . 10 gaming, and software-as-a-service (SaaS) single point of failure, designed to contain
applications are just some of the forces large amounts of unstructured data at low
Summary. . . . . . . . . . . . . . . . . . . . . . . . 11
driving data centers to take a new look cost and accessible via a REST API. In
Learn More. . . . . . . . . . . . . . . . . . . . . . 12 Swift, every object has its own URL.
at how data is stored.
Appendix A –
Additional Resources. . . . . . . . . . . . . 12 Swift meets these types of demands, Swift can easily scale as the cluster grows
from small deployments for storing virtual in the number of requests and data
machine (VM) images, to mission-critical capacity. The proxy servers that handle
storage clusters for high-volume web incoming API requests scale in a near-
sites, to mobile application development linear fashion by adding more servers. The
environments, custom file-sharing system uses a shared-nothing approach
applications, big data analytics, and and employs the same proven techniques
private storage Infrastructure as a that have been used to provide high
Service (IaaS). Swift originated in a availability by many web applications. To
large-scale production environment scale out storage capacity, nodes and/or
at RackSpace, so it was designed for drives are added to the cluster. Swift can
large-scale, durable operations. easily expand from a few gigabytes on a
couple machines to dozens of petabytes
on thousands of machines scattered
around the planet, simply by adding
hardware. Swift automatically incorpo-
rates the new devices into its resources.
2Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Since all content in the Object Store is Python.* Swift runs on off-the-shelf Cost can deter using public cloud storage
available via a unique URL, Swift can easily Linux* distributions, including Fedora,* services. Public cloud storage costs
serve content to modern web applications, RedHat Enterprise Linux* (RHEL), include per-GB pricing and data transit
and it also becomes very straightforward openSUSE*, and Ubuntu.* charges, which grow rapidly for larger
to either cache popular content locally or storage requirements. But, with the
From a hardware perspective, Swift is
integrate it with a CDN, such as Akamai.* declining costs of data center servers
designed from the ground up to handle
and drives and the emergence of new
Durability failures, so that reliability of individual
high-efficiency microservers based on
components is less critical. Thus, Swift
Downtime is costly. Lack of content can the same x86 instruction set architec-
clusters can run on a broad range of
affect a wide range of users, preventing ture as the other servers in the data
multi-socket servers and low-cost,
them from doing their work or using the center, the total cost of ownership (TCO)
high-performance microservers, with
service to which they subscribe. Swift can for a Swift cluster can be on par with
commodity HDDs and/or SSDs. Hardware
withstand hardware failures without any AWS S3 for a small cluster and much
quality and configuration can be chosen to
downtime and provides operations teams less than AWS S3 for a large cluster.
suit the tolerances of the application and
the means to maintain, upgrade, and
the ability to replace failed equipment. In addition, the network latency to
enhance a cluster while in flight. To
public storage service providers may be
achieve this level of durability, Swift Compatibility and Familiarity unacceptable, especially for time-based
distributes objects in triplicate across the
If an operator considers moving from a content. A private deployment can
cluster. A write must be confirmed in two
public cloud service to an internal installa- provide lower-latency access to storage.
of the three locations to be considered
tion, the method used by services to
successful. Auditing processes maintain
data integrity, and replicators ensure a
access the storage becomes more Overview of the Swift Architecture
important. Access to the Swift Object The Swift Object Storage architecture is
sufficient number of copies across the
Storage system is through a REST API that distributed across a cluster to prevent any
cluster, even if a storage device fails.
is similar to the Amazon.com S3* API and single point of failure and to allow easy
Swift also can define failure zones. Failure compatible with the Rackspace Cloud horizontal scalability. A number of periodic
zones allow a cluster to be deployed Files* API. This means that: (a) applications processes perform housekeeping tasks on
across physical boundaries, each of which currently using S3 can use Swift without the data store. The most important of
could individually fail. For example, a major re-factoring of the application code; these are the replication services, which
cluster could be deployed across several and (b) applications taking advantage of ensure consistency and availability
nearby data centers, enabling it to survive both private and public cloud storage can throughout the cluster. Other periodic
multiple datacenter failures. do so, as the APIs are comparable. processes include auditors, updaters,
Because Swift is compatible with public and reapers.
Flexibility
cloud services, developers and systems Authentication is handled through
Proprietary solutions can drive up
architects can also take advantage of a configurable Web Server Gateway
licensing costs. Swift is licensed freely
rich ecosystem of available commercial Interface (WSGI) middleware—usually
under the Apache 2 open source license,
and open-source tools for these object the Keystone* Architecture.
ensuring no vendor lock-in, providing
storage systems.
community support, and offering a
large ecosystem. Public or Private Cloud
Choice of Components Not every organization can—or should—
use a public cloud to store sensitive
Proprietary storage solutions offer
information. For private cloud infrastruc-
turnkey benefits, but at a higher purchase
tures, Swift enables organizations to reap
price and little or no choice of components
the benefits of cloud-based object
within the solution. Open source Swift
storage, while retaining control over
software is built on proven, industry-wide
network access, security, and compliance.
components that work in large-scale
production environments, such as rsync,
MD5, SQLite, memcached, xfs, and
3Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Primary Objective: High Availability and Upload Download
Fast Access to a Wide Variety of Data Through a REST API, a client makes an Through the API, a client requests an
Swift provides highly durable storage for HTTP request to put an object into an object from the data store. The object’s
large data stores containing a wide variety existing Container. The Proxy passes the name and partition location are identified
of data that a large number of clients can request to the system. Using the Account in the same manner, and a lookup in the
read and write. and Container services, the logical name Ring reveals which storage node contains
of the Object is identified. The Ring then the Partition. A request is made to one of
Theory of Operation finds the physical location of the partition the storage nodes to fetch the object. If
Swift software architecture includes on which the object is stored. Then, the the download fails, requests are made to
the following elements: system sends the data to each storage the other nodes. Streamed data to or
• Proxies: Handle all incoming node, where it is placed in the appropriate from an object store is never spooled by
API requests. Partition. At least two of the three writes the Proxy.
must be successful before the client is
• Objects: The data being stored. notified that the upload completed Replication
• Containers: An individual database that successfully. If storage areas are not In order to ensure there are three copies
contains a list of object names. available, the Proxy requests a handoff of the data everywhere, replicators
server from the Ring and places it there. continuously examine each Partition. For
• Accounts: An individual database that
Finally, the system asynchronously each local Partition, the replicator checks
contains the list of Container names.
updates the Container database to reflect copies in other Zones for any differences.
• Ring: A map of logical names of data that there is a new object in it. When it discovers differences among
(defined by the Account/Container/ Partitions, it copies the latest version
Object) to locations on particular storage across the Zones.
devices in the cluster.
• Partition: A “bucket” of stored data.
Multiple Partitions contain the Objects,
Container databases, and Account
databases. Swift replication services
move and copy partitions to maintain
high availability of the data. 140
• Zone: A unique isolated storage area Upload Requires Download from
contained in the cluster, which might Quorum Single Node 120
be a single disk, a rack, or entire data
Load Balancer
center. Swift tries to maintain copies of 100
partitions across three different Zones
in the cluster. Proxy Proxy Proxy
Node Node Node 80
• Auth: An optional authorization
service, typically run within Swift
60
as WSGI middleware.
To see how these elements interact, let’s
40
look at a couple scenarios.
Storage Storage Storage Storage Storage
Node Node Node Node Node 20
Storage Storage Storage Storage Storage
Node Node Node Node Node
Storage Storage Storage Storage Storage
Node Node Node Node Node
Zone 1 Zone 2 Zone 3 Zone 4 Zone 5
Figure 1. Swift Cluster Software Architecture.
4 OpenStack Object Storage
Stores container databases, account databases, and stored objectsConfiguration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Other Housekeeping Services •S
torage node(s) – One or more server(s) This tier typically comprises a collection
Other housekeeping services run to with local storage that run Account, of Intel Xeon E3 processor-based
maintain system durability and keep Container, and Object services. single-socket servers or Intel® Xeon® E5
files updated. For more information on The Proxy and Auth nodes face the processor-based dual-socket servers.
1400
housekeeping services, refer to the Swift public network, with a private switch Machines in this tier use a moderate
Upload Requires Download from amount of RAM and are network I/O
documentation (swift.openstack.org). isolating
Quorum Singlethe storage nodes from the
Node 1200
public network. intensive. Proxy nodes should be
Load Balancer
Configuration and Deployment provisioned with at least two 10 Gbps
1000 1062
Swift architecture comprises many Access Tier network interfaces, or multiples thereof.
Large-scale One faces the incoming requests on the
services running on various hardwareProxy
Proxy Proxy deployments often isolate
nodes. Depending on theNode Node
infrastructure, an “Access
Node Tier” to field incoming API public network and the other accesses
800
business processes, and available hard- requests, move data in and out of the the private network to the object
ware, some services can reside on the system, provide front-end load balancers, storage nodes. Depending on 688 the
600
same hardware. Secure Sockets Layer* (SSL) terminators, expected activity level, other nodes
authentication services, and to run the only facing the public network can have
Swift Topology 400 432
single or multiple interfaces.
proxy server processes. Having these
Swift software runs on many servers/ servers in their own tier enables read/
Storage Storage Storage Storage Storage
nodes (see Figure 2):Node
Node Node write access
Node to be scaled out independent-
Node 200
ly of storage capacity. As this is an HTTP 126
• Proxy node(s) – Server(s)
Storage Storage that runStorage Storage Storage
Node Node Node
addressable
Node
storage service,
Node
a load 0
Proxy services.
balancer can be incorporated into the Performance (Ops/sec)
Storage
• Auth Storage node that
node – an optionally Storage accessStorage
tier. Storage (Higher is better)
Node Node Node Node Node
runs the Auth service separately from Intel® Atom™ Processor S1260 Intel®
Zone 1 Zone
the Proxy services. 2 Zone 3 Zone 4 Zone 5
Intel® Xeon® E3-1230L v3
OpenStack Object Storage
Stores container databases, account databases, and stored objects
Storage Nodes
Private
Public Switch
Switch
Auth Node Proxy Node
Internet
Figure 2. Swift Topology.
5Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Factors to Consider Factors to Consider Performance Considerations
For most publicly facing deployments as The latest generation of Intel Atom Overall Strategy
well as private deployments on a wide- processor-based microservers and
A simple deployment is to install the Proxy
reaching corporate network, SSL should microservers built on the new Intel Xeon
Services on their own servers and all of
be considered. However, SSL adds E3 processor work well for storage
the Storage Services across the storage
significant server processing load, unless servers. With up to eight-thread multi-
servers. This allows easy deployment of
the server processor has built-in hardware processing, they are responsive and
10 Gbps networking to the proxy and 1
support for encryption/decryption, such highly energy efficient, and cost less
Gbps to the storage servers. It also helps
as Intel® Advanced Encryption Standard— than higher-end multi-socket servers.
keep load balancing to the proxies more
New Instructions (Intel® AES-NI). More With up to 16 PCIe* ports, they also
manageable. Storage services scale as
capacity in the access layer might be support an ample number of Intel®
storage servers are added, and it’s easy
needed if hardware support is lacking. Solid-State Drives (Intel® SSDs) or
to scale overall API throughput by adding
SSL may not be required for private traditional HDDs, while delivering the
more Proxies.
deployments on trusted networks. performance necessary to service
requests, even under heavy workloads. If you need more throughput to either
Storage Nodes Account or Container Services, you can
Swift does not use Redundant Array of
Storage servers should contain an equal deploy the services on their own servers
Inexpensive Disks (RAID); each request for
amount of capacity on each server. and use faster SAS or SSD drives to get
an object is handled by a single disk.
Depending on your objectives and needs, quicker disk I/O.
Therefore, disk performance impacts
these can be distributed across servers,
response rates, and Intel SSDs should be Nodes
racks, and even data centers.
implemented for ultra-fast responsiveness In the front-end, the Proxy Services are
Storage nodes use a reasonable amount when desired. CPU- and network I/O-intensive. Select
of memory and CPU. Metadata needs to
To achieve apparent higher throughput, hardware accordingly to handle the
be readily available to quickly return
the object storage system is designed amount of expected traffic on these
objects. The object stores run services
to support concurrent uploads and nodes. More cores can service more
not only to field incoming requests from
downloads. Multi-core processors, like requests. As already pointed out, if you
the Access Tier, but to also run replica-
the Intel Atom processor and Intel Xeon are using 10 Gbps networking to the
tors, auditors, and reapers. Object stores
E3 processor, offer multi-processing for proxy, or are terminating SSL traffic at the
can be provisioned with a single 1 Gbps
concurrent operations. However, the proxy, greater CPU power will be required
or 10 Gbps network interface, depending
network I/O capacity should match the if the server does not support hardware-
on the expected workload and desired
desired concurrent throughput needs enhanced encryption and decryption.
performance. More interfaces might be
for reads and writes. The Object, Container, and Account
appropriate for servers with more
processor cores. Services (collectively, the Storage
Services) are disk- and network
Currently 2 TB or 3 TB Serial Advance
I/O-intensive. As already mentioned,
Technology Attachment (SATA) disks
SSDs should be considered for maximum
deliver good price/performance value.
responsiveness from storage nodes.
Desktop-grade drives offer affordable
As shown later in this guide, Intel Atom
performance when service and support
processor- and Intel Xeon E3 processor-
are responsive to handle drive failures;
based microservers are quite capable as
enterprise-grade drives are an option
storage nodes.
when this is not the case.
Load balancing and network design is left
as an exercise to the reader, but this is a
very important part of the cluster, so time
should be spent designing the network for
a Swift cluster.
6Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Ring Configuration General Server Configuration Intel completed studies on storage node
It is important to determine the number Swift uses paste.deploy (http://python- performance from microservers built with
of partitions that will be in the Ring prior paste.org/deploy/) to manage server the latest generation of Intel Atom proces-
to configuring the cluster. Consider a configurations. The configuration process sors and Intel Xeon E3 processors. The
minimum of 100 partitions per drive to for paste.deploy is rather particular; it is results show the value of these latest
ensure even distribution across the drives. best to review the documentation to be generation microservers to enable
Decide the maximum number of drives the sure you configure servers optimally. high-performance and energy-efficient
cluster will contain, multiply that by 100, Swift object storage nodes.
and then round up to the nearest power Memcached Considerations
Because SSDs offer much faster response
of two. Memcached is an open-source,
times over traditional spinning HDDs, Intel
multi-threaded, distributed, Key-Value
For example, for a cluster with a maximum also compared storage node performance
caching solution that can cache “hot”
of 5,000 drives, the total number of between HDDs and Intel SSDs. The results
data and reduce costly trips back to
partitions would be 500,000, which is support a strong argument for SSDs
the database to read data from disk. It
close to 2,19 rounded up. where storage responsiveness is desired.
implements a coherent in-memory RAM
Keep in mind, the more partitions, the cache that scales across a cluster of COSBench Platform
more work the replicators and other servers. Several Swift Services rely on The Swift storage benchmarks reported
backend jobs have to do, and the more Memcached for caching certain types here used the COSBench* platform to
memory the Rings consume. The goal is to of lookups, such as auth tokens and create loads and gather performance
find a good balance between small rings container/account existence. Memcached data. COSBench is an open source,
and maximum cluster size. exploits the fact that the typical access Intel-developed benchmarking tool to
time for DRAM is orders of magnitude measure Cloud Object Storage perfor-
It’s also necessary to decide on the
faster than disk access times, thus mance on services like Amazon S3 and
number of replicas of the data to store.
enabling considerably lower latency OpenStack Swift. COSBench is freely
Three (3) is the current recommendation,
and much greater throughput. distributed under the Apache V2 license
because it has been tested in the
industry. The higher the number, the Swift does not cache actual object data. and is available at https://github.com/
more storage is used, and the less likely Memcached should run on any servers intel-cloud/cosbench. You can learn more
you are to lose data. that have available RAM and CPU about COSBench at https://github.com/
capacity. The memcache_servers config intel-cloud/cosbench/blob/master/
Finally, you’ll need to determine the cosbench-introduction.pdf.
option in the proxy-server.conf file
number of Zones in the cluster. Five (5) is
should contain all memcached servers.
the current recommendation according to
For more information, see Intel’s
Swift documentation. Having at least five
“Configuration and Deployment Guide
zones is optimal when failures occur. Try Next-generation Intel® Atom™
For Memcached on Intel® Architecture.”
to configure the zones in as high a level as
possible to create as much isolation as processor codenamed Avoton
Storage Server Benchmarking
possible. Consider physical location, power
for Swift delivers better performance
availability, and network connectivity to
Object storage in cloud infrastructure can
help determine isolated Zones. For and performance/watt compared
easily scale quickly as storage demands
example, in a small cluster you might
isolate the Zones by cabinet, with each
rapidly increase, especially considering the to previous-generation Intel
growing need to contain video and other
cabinet having its own power and network
large media. The power demands at scale Atom processor.
connectivity. The Zone concept is very
become critical and more strongly drive
abstract; use it to best isolate your data
hardware choices for storage nodes.
from failure.
Thus, energy-efficient, high-performance
servers, such as microservers, become
more attractive in cloud deployments.
7Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Intel® Atom™
Intel® Xeon® Intel® Xeon® Intel® Xeon®
Processor Intel® Atom™
Configuration E3-1220L v3 E3-1230L v3 E3-1265L v3
(codenamed Processor S1260
Processor Processor Processor
Avoton)
Platform See Table 2 See Table 2 See Table 2 See Table 2 Codenamed DoubleCove
S1200RP.86B. S1200RP.86B. S1200RP.86B. EDVLINT1.86B.0018.
BIOS
01.01.2003 01.01.2003 01.01.2003 D05.1306021543_MPK
C state and p states dis-
BIOS Setting Cstate off, Intel® HT on Cstate off, Intel® HT on Cstate off, Intel® HT on abled “Active Refresh” Default
“CKE Power Down”
CPU GHz 1.1 1.8 2.5 2.4 2
# Cores 2 4 4 8 2
# Nodes 3 3 3 3 3
Sockets/Node 1 1 1 1 1
Memory (GB)/Node 32 32 32 16 8
# DIMMs 4 4 4 2 2
DIMM size (GB) 8 8 8 8 4
# Channels 2 2 2 2
Memory Speed 1600 1600 1600 1600 1333
OS Ubuntu Server 12.04 Ubuntu Server 12.04 Ubuntu Server 12.04 Ubuntu Server 12.04 Ubuntu Server 12.04
NIC Port Speed 10 Gbps 10 Gbps 10 Gbps 1 Gbps 1 Gbps
NIC Ports/Node 1 1 1 4 2
NIC Location Niantic* 10G Niantic* 10G Niantic* 10G On board On board
Boot Drive
Quantity 1 1 1 1 1
RPM/SSD SSD SSD SSD SSD SSD
Size 160 GB 160 GB 160 GB 160 GB 160 GB
SATA Yes Yes Yes Yes Yes
Storage Drive (SSD)
Quantity 5 5 5 3 1
RPM/SSD SSD SSD SSD SSD SSD
Size 250 GB 250 GB 250 GB 160 GB 160 GB
SATA Speed 3 Gb/s 3 Gb/s 3 Gb/s 3 Gb/s 3 Gb/s
Storage Drive (HDD)
Quantity 5 5 5 3 NA
RPM/SSD HDD HDD HDD HDD NA
Size 1TB 1TB 1TB 1TB NA
SATA Speed 3 Gb/s 3 Gb/s 3 Gb/s 3 Gb/s NA
Table 1. Storage Server Benchmark Configurations.
8Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
System Under Tests (SUT) Results In large cloud data centers, where server
Configurations Benchmark results (see Figure 3) clearly
1,2 counts can scale quickly, conserving power
The microservers systems under test show that the new Intel Atom processor by using energy-efficient hardware
(SUT) used in the benchmarks comprise a codenamed Avoton-based microserver becomes more critical. Microservers built
range of servers offering important outperforms the previous-generation on the Intel Atom processor codenamed
characteristics needed in today’s cloud Intel® Atom™ Processor S1260 server by Avoton deliver the highest efficiency with
storage solutions. Only the storage over 5X, and does so more efficiently— good performance, making this configura-
servers were tested in these studies; more than 3.5X performance/watt tion an excellent balance where perfor-
Proxy server, while part of the configura- improvement. The latest generation Intel mance, price, and power consumption are
tion, was not measured. Table 1 lists Xeon E3 processor-based microservers important factors.
the storage server configurations. Five offer both performance above the latest
workloads (Table 3) stressed different generation Intel Atom processor-based
aspects of the configurations. microserver and good energy efficiency.
Component Supported Specification Workload Test Rationale
CPU Cores 8 LrgRead Randomly read objects Stress Read IO bandwidth
Core Frequency Base / Turbo 2.4 GHz/2.6 GHz of 1 MB, 100 MB, from the object store server
and 1 GB
Memory Channels 2
DIMMs/Channel 2 LrgWrite Randomly Write Stress Write IO bandwidth
objects of 1 MB, to the object store server
Memory Type DDR3L 100 MB, and 1 GB
Memory Frequency 1600
SmlWrite Randomly write objects Stress the container servers
Maximum Memory Capacity 32 GB of 100 B, 1 KB, and 10 ability to track user file data
PCIe* Lanes – Maximum 16 KB from multiple clients in the SQL-lite database, by
applying a high write rate,
PCIe Controllers 4 which is replicated across
Gb Ethernet Ports/Speed 4, 2.5 Gb Ethernet 3 servers
SATA 3 Ports 2 SmlRead Randomly read objects Stress the Proxy servers
of 100 B, 1 KB, and 10 ability to process user
SATA 2 Ports 4
KB from multiple clients file data
USB Ports 4
LrgRWMix Present a randomly Test for locks inside the
Table 2. Storage System Under Test (SUT) Platform Specifications. selected R/W mix of databases and other stores
90/10 of 1 MB, 100 MB, due to concurrent reads and
and 1 GB files sizes writes to the system
Table 3. Benchmark Workloads.
9Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
These results were achieved using Intel Tuning and Optimization On systems that have more cores and
SSDs for both boot and storage drives. more memory, you can run more workers.
General Service Tuning
Additionally, the benchmarks revealed the Raising the number of workers and
impact SSDs have on performance and Most services support either a worker or lowering the maximum number of clients
efficiency compared to SATA 7200 rpm concurrency value in the settings. This serviced per worker can lessen the impact
and Serial Attached SCSI (SAS) 10k rpm allows the services to make effective use of CPU-intensive or stalled requests.
storage drives. SSDs performed as much of the cores available. A good starting
point to set the concurrency level for the The above configuration settings are
as 4.9X faster, as shown Table 4, with
proxy and storage services is twice (2X) suggestions; test your settings and adjust
more than 5X efficiency.
the number of cores available. If more to ensure the best utilization of CPU,
Finally, benchmarks offered a view into than one service shares a server, then network connectivity, and disk I/O. Always
performance benefits of adding storage some experimentation may be needed to refer to the Swift documentation for the
drives to each storage node as shown in find the best balance. most recent recommendations.
Table 5, with nearly linear scaling for each
drive added. Set the max_clients parameter to adjust Filesystem Considerations
the number of client requests an individu- Swift is rather filesystem agnostic; the
These benchmarks reveal the capability of al worker accepts for processing. The only requirement is the filesystem must
latest-generation Intel Atom processor- fewer requests being processed at one support extended attributes (xattrs). XFS
and Intel Xeon processor-based microservers time, the less likely a request will consume has proven to be the best choice. Other
when used for object storage servers in a the worker’s CPU time or block the OS. filesystem types should be thoroughly
Swift cluster, making these attractive The more requests being processed at tested prior to deployment.
systems for scalable cloud infrastructure. one time, the more likely one worker can
utilize network and disk capacity.
1400 4.0
1200 3.5 3.59
1160 3.31 3.26
1062 3.0
1000
2.5
800
2.0
688
600
1.5 1.65
400 432
1.0
1.0
200 .5
126
0 0
Performance (Ops/sec) Power Efficiency (Performance/Watt)
(Higher is better) (Higher is better)
Intel® Atom™ Processor S1260 Intel® Xeon® E3-1220L v3 Processor Intel® Atom™ Processor (Avoton)
Intel® Xeon® E3-1230L v3 Processor Intel® Xeon® E3-1265L v2 Processor
Figure 3. Benchmark Results.
tored objects
10Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Performance (Ops/sec)
Processor Intel® SSD 7.2k HDD 10k SAS HDD Improvement
Intel® Atom™ Processor (Avoton) 688 140 341 4.9X/2.1X
Intel® Xeon® E3-1220L v3 Processor 432 145 NA 3X
Intel® Xeon® E3-1265L v3 Processor 1160 238 NA 4.9X
Intel® Xeon® E3-1230L v3 Processor 1062 327 NA 3.2X
Efficiency (Performance/Watt)
Intel Atom Processor (Avoton) 3.59 0.66 1.56 5.4X/2.3X
Intel Xeon E3-1220L v3 Processor 1.65 0.46 NA 3.6X
Intel Xeon E3-1265L v3 Processor 3.26 0.64 NA 5.1X
Intel Xeon E3-1230L v3 Processor 3.31 0.93 NA 3.6X
Table 4. SSD vs. HDD Performance Results.
Number of Drives
Metric 1 2 3
Performance (Operations/sec) 184 327 638
Efficiency (Performance/Watt) 0.95 1.61 3.05
Table 5. Storage Drive Scaling Results (Intel® Atom™ processor codenamed Avoton). (Higher is Better)
Summary
Open source Swift object storage software provides a solution for companies interested
in creating their own private cloud storage service or building a service for other
production purposes. Swift is easy to use, and it supports web-scale storage services
compatible with web applications used today. Swift is provided under the Apache 2 open
source license, is highly scalable, extremely durable, and runs on industry-standard
hardware, such as microservers based on the latest generation of Intel Atom processor
or Intel Xeon E3 processors. Swift also has a compelling set of tools available from third
parties and other open source projects.
Server power consumption has become a critical driving factor in data centers. In
benchmarks, microservers based on the latest generation of Intel Atom processor and
Intel Xeon E3 processors deliver exceptional performance and performance/watt as
Swift storage servers. These multi-core microservers outperformed previous genera-
tion, highly efficient Intel Atom processor S1260 servers. With massive object storage
deployments typical today, the latest generation of Intel Atom processor- and Intel Xeon
E3 processor-based microservers offer attractive choices for Swift cluster deployments.
11Configuration and Deployment Guide For OpenStack* Swift* Object Storage
on Intel® Atom™ Processor and Intel® Xeon® Processor Microservers
Learn More
For more information, visit the home of Swift at the OpenStack web site www.openstack.org and
SwiftStack*, a Swift object storage service provider www.swiftstack.com.
For more information on Intel Atom processor-based servers, visit www.intel.com/content/www/us/en/processors/
atom/atom-processor.html.
Appendix A – Additional Resources
http://docs.openstack.org/developer/swift/ for Swift documentation.
http://docs.openstack.org/developer/swift/deployment_guide.html for Swift deployment, configuration, and tuning information.
“Configuration and Deployment Guide For Memcached on Intel® Architecture,” Intel Corporation
1
oftware and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using
S
specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to
assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
2
Configurations: see Tables 1 and 2. For more information go to http://www.intel.com/performance.
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual
performance.
Intel® Hyper-Threading Technology available on select Intel® Core™ processors. Requires an Intel® HT Technology-enabled system. Consult your PC manufacturer. Performance will vary depending on the
specific hardware and software used. For more information including details on which processors support HT Technology, visit http://www.intel.com/info/hyperthreading.
Any software source code reprinted in this document is furnished under a software license and may only be used or copied in accordance with the terms of that license.
NFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL
PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY
WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO
FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A “Mission Critical Application” is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL’S PRODUCTS
FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS,
AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS’ FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY
CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR
WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or
“undefined”. Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change
without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available
on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm
Copyright © 2013 Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Atom, and Intel Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others. Printed in USA 0913/DF/HBD/PDF Please Recycle 329542-001USYou can also read
