FlashAbacus: A Self-Governing Flash-Based Accelerator for Low-Power Systems
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
FlashAbacus: A Self-Governing Flash-Based
Accelerator for Low-Power Systems
Jie Zhang and Myoungsoo Jung
Computer Architecture and Memory Systems Laboratory
School of Integrated Technology,Yonsei University,
http://camelab.org
arXiv:1805.02807v1 [cs.AR] 8 May 2018
Abstract constraints of most embedded systems are approximately a
Energy efficiency and computing flexibility are some of the few watts [8]. To satisfy this low power requirement, ASICs
primary design constraints of heterogeneous computing. In and FPGAs can be used for energy efficient heterogeneous
this paper, we present FlashAbacus, a data-processing accel- computing. However, ASIC- or FPGA-based accelerators
erator that self-governs heterogeneous kernel executions and are narrowly optimized for specific computing applications.
data storage accesses by integrating many flash modules in In addition, their system usability and accessibility are of-
lightweight multiprocessors. The proposed accelerator can ten limited due to static logic and inflexible programming
simultaneously process data from different applications with tasks. Alternatively, embedded multicore processors such as
diverse types of operational functions, and it allows multiple the general purpose digital signal processor (GPDSP), em-
kernels to directly access flash without the assistance of a bedded GPU and mobile multiprocessor can be employed
host-level file system or an I/O runtime library. We prototype for parallel data processing in diverse low power systems
FlashAbacus on a multicore-based PCIe platform that con- [25, 35].
nects to FPGA-based flash controllers with a 20 nm node Although the design of low-power accelerators with em-
process. The evaluation results show that FlashAbacus can bedded processors opens the door to a new class of hetero-
improve the bandwidth of data processing by 127%, while geneous computing, their applications cannot be sufficiently
reducing energy consumption by 78.4%, as compared to a well tuned to enjoy the full benefits of the data-processing
conventional method of heterogeneous computing. acceleration; this is because of two root causes: i) low pro-
cessor utilization and ii) file-associated storage accesses.
1. Introduction Owing to data transfers and dependencies in heterogeneous
computing, some pieces of code execution are serialized, in
Over the past few years, heterogeneous computing has un- turn limiting the processing capability of the low-power ac-
dergone significant performance improvements for a broad celerators. In addition, because, to the best of our knowl-
range of data processing applications. This was made possi- edge, all types of accelerators have to access external storage
ble by incorporating many dissimilar coprocessors, such as when their internal memory cannot accommodate the target
general-purpose graphics processing units (GPGPUs) [50] data, they waste tremendous energy for storage accesses and
and many integrated cores (MICs) [14]. These accelerators data transfers. External storage accesses not only introduce
with many cores can process programs offloaded from a host multiple memory copy operations in moving the data across
by employing hundreds and thousands of hardware threads, physical and logical boundaries in heterogeneous comput-
in turn yielding performance than that of CPUs many orders ing, but also impose serial computations for marshalling the
of magnitude [5, 21, 47]. data objects that exist across three different types of com-
While these accelerators are widely used in diverse het- ponents (e.g., storage, accelerators, and CPUs). Specifically,
erogeneous computing domains, their power requirements 49% of the total execution time and 85% of the total system
render hardware acceleration difficult to achieve in low energy for heterogeneous computing are consumed for only
power systems such as mobile devices, automated driving data transfers between a low-power accelerator and storage.
systems, and embedded designs. For example, most modern In this paper, we propose FlashAbacus, a data process-
GPGPUs and MICs consume as high as 180 watts and 300 ing accelerator that self-governs heterogeneous comput-
watts per device [1, 14], respectively, whereas the power ing and data storage by integrating many flash modules
This paper is accepted by and will be published at 2018 EuroSys. This in lightweight multiprocessors to resemble a single low-
document is presented to ensure timely dissemination of scholarly and power data processing unit. FlashAbacus employs tens of
technical work.
1ǀŝͿhƐĞƌƐƉĂĐĞƚŽ ǀͿĐĐĞůĞƌĂƚŽƌ ĂƚĂͲŝŶƚĞŶƐŝǀĞƉƉůŝĐĂƚŝŽŶ
ĂĐĐĞůĞƌĂƚŽƌƐƚĂĐŬ ƐƚĂĐŬƚŽĚĞǀŝĐĞ WK^/y/ͬK;ĨƌĞĂĚ͕ĨǁƌŝƚĞͿ
❸ hƐĞƌ^ƉĂĐĞ /ͬKZƵŶƚŝŵĞ ĐĐ͘ZƵŶƚŝŵĞ
❷ DWƐ
DĂŝŶWh ĂĐŚĞ ĨŝůĞͺŽƉĞƌĂƚŝŽŶƐ
&ŝůĞ^LJƐƚĞŵ
ZD
ZD
ZD EŽƌƚŚďƌŝĚŐĞ DĞŵŽƌLJ ❷
ŝŝŝͿ^ƚŽƌĂŐĞƐƚĂĐŬ ďůŽĐŬͺĚĞǀŝĐĞͺŽƉĞƌĂƚŝŽŶƐ ĐĐ͘ƌŝǀĞƌ
ƚŽƵƐĞƌƐƉĂĐĞ ❶ ĐĐĞůĞƌĂƚŽƌ
,ƌŝǀĞƌ,ŽƐƚ &hϬ &hϭ &hϰ &hϱ &ůĂƐŚďĂĐŬďŽŶĞ
tWϬ >tWŶ
&ůĂƐŚ
D
&D
ƌŝĚŐĞ ĚĂƉƚĞƌ
dŝĞƌͲϭEĞƚǁŽƌŬ ĂƌĚ
&ůĂƐŚ
ĨůĂƐŚ
ƚƌůĞƌ
&W'Components Specification
Working Typical Est. Lastly, the flash controllers are built on the 2 million system
frequency power B/W logic cells of Virtex Ultrascale FGPA [78]. The important
LWP 8 processors 1GHz 0.8W/core 16GB/s
L1/L2 cache 64KB/512KB 500MHz N/A 16GB/s
characteristics of our prototype are presented in Table 1.
Scratchpad 4MB 500MHz N/A 16GB/s
Memory DDR3L, 1GB 800MHz 0.7W 6.4GB/s
SSD 16 dies, 32GB 200MHz 11W 3.2GB/s
PCIe v2.0, 2 lanes 5GHz 0.17W 1GB/s 3. High-level View of Self-Governing
Tier-1 crossbar 256 lanes 500MHz N/A 16GB/s
Tier-2 crossbar 128 lanes 333MHz N/A 5.2GB/s Figure 3a illustrates a kernel execution model for conven-
tional hardware acceleration. In prologue, a data process-
ing application needs to open a file and allocate memory
Table 1: Hardware specification of our baseline. resources for both an SSD and an accelerator. Its body it-
erates the code segments that read a part of file, transfer it to
the accelerator, execute a kernel, get results from the accel-
L2 cache. Note that all LWPs share a single memory address erator, and write them back to the SSD. Once the execution
space, but have their own private L1 and L2 caches. of body loop is completed, the application concludes by re-
Network organization. Our hardware platform uses a par- leasing all the file and memory resources. This model oper-
tial crossbar switch [62] that separates a large network into ates well with traditional manycore-based high-performance
two sets of crossbar configuration: i) a streaming cross- accelerators that have thousands of hardware threads. How-
bar (tier-1) and ii) multiple simplified-crossbars (tier-2). ever, in contrast to such traditional accelerators, the mem-
The tier-1 crossbar is designed towards high performance ory space of low-power accelerators is unfortunately limited
(thereby integrating multiple LWPs with memory modules), and difficult to accommodate all data sets that an application
whereas throughputs of the tier-2 network are sufficient for requires to process. Thus, low-power accelerators demand
the performances of Advanced Mezzanine Card (AMC) and more iterations of data transfers, which in turn can signif-
PCIe interfaces exhibit [28]. These two crossbars are con- icantly increase I/O stack overheads. Furthermore, a small
nected over multiple network switches [70]. memory size of the low-power accelerators can enforce a
Flash backbone. Tier-2 network’s AMC is also connected single data-processing task split into multiple functions and
to the FPGA Mezzanine Card (FMC) of the backend stor- kernels, which can only be executed by the target accelerator
age complex through four Serial RapidIO (SRIO) lanes in a serial order.
(5Gbps/lane). In this work, the backend storage complex
is referred to as flash backbone, which has four flash chan-
nels, each employing four flash packages over NV-DDR2 3.1 Challenge Analysis
[58]. We introduce a FPGA-based flash controller for each For better insights on the aforementioned challenges, we
channel, which converts the I/O requests from the processor built a heterogeneous system that employs the low-power
network into the flash clock domain. To this end, our flash accelerator described in Section 2.2 and an NVMe SSD [32]
controller implements inbound and outbound “tag” queues, as external storage instead of our flash backbone.
each of which is used for buffering the requests with min- Utilization. Figures 3b and 3c show the results of our perfor-
imum overheads. During the transition device domain, the mance and utilization sensitivity studies in which the frac-
controllers can handle flash transactions and transfer the cor- tion of serial parts in kernel executions enforced by data
responding data from the network to flash media through the transfers were varied. As the serial parts of a program in-
SRIO lanes, which can minimize roles of flash firmware. crease, the throughput of data-processing significantly de-
It should be noted that the backend storage is a self- creases (Figure 3b). For example, if there exists a kernel
existence module, separated from the computation complex whose fraction of serialized executions is 30%, the perfor-
via FMC in our baseline architecture. Since flash backbone mance degrades by 44%, on average, compared to the ideal
is designed as a separable component, ones can simply re- case (e.g., 0%), thus the accelerator becomes non-scalable
place worn-out flash packages with new flash (if it needs). and the full benefits of heterogeneous computing are not
Prototype specification. Eight LWPs of our FlashAbacus achieved. Poor CPU utilization is the reason behind perfor-
operate with a 1GHz clock and each LWP has its own private mance degradation (Figure 3c). For the previous example,
64KB L1 cache and 512KB L2 cache. The size of the 8-bank processor utilization is less than 46%; even in cases where
scratchpad is 4MB; DDR3L also consists of eight banks with the serial parts take account for only 10% of the total kernel
a total size of 1GB. On the other hand, the flash backbone executions, the utilization can be less than at most 59%.
connects 16 triple-level cell (TLC) flash packages [48], each Storage accesses. We also evaluate the heterogeneous sys-
of which has two flash dies therein (32GB). Each of the four tem by performing a collection of PolyBench [57] and de-
flash packages is connected to one of the four NV-DDR2 compose the execution time of each application into i) SSD
channels that work on ONFi 3.0 [58]. The 8KB page read latency to load/store input/output data, ii) CPU latency that
and write latency are around 81 us and 2.6 ms, respectively. host storage stack takes to transfer the data, and iii) acceler-
4Execution time breakdown
ĨŽƉĞŶ;Ϳ /ͬKZƵŶƚŝŵĞ Serial 50% 40% 30% Serial 50% 40% 30% Accelerator SSD Accelerator SSD
ŵĂůůŽĐ;Ϳ ĐĐ͘ZƵŶƚŝŵĞ ratio 20% 10% 0% ratio 20% 10% 0% Host storage stack Host storage stack
Energy breakdown
5
(GB/s)
100
Core Utilization (%)
ĐĐͲDĂůůŽĐ;Ϳ 1.0 1.0
ůŽŽƉ 4 80 0.8
WƌŽůŽŐƵĞ
ĨƌĞĂĚ;Ϳ 3 0.6
60 0.5
Throughput
ĐĐͲDĞŵĐƉLJ;Ϳ 2 0.4
ĐĐͲŬĞƌŶĞů;Ϳ 40 0.2
ĨƌĞĞ;Ϳ
ĐĐͲDĞŵĐƉLJ;Ϳ
1 0.0 0.0
ĐĐͲ&ƌĞĞ;Ϳ 20
0
B X
M N
B X
M N
2D ICG
2D ICG
SY VT
SY VT
RK
GE3MM
M
VA I
FD R
TD
RK
GE3MM
M
VA I
FD R
TD
3 co r r e
4 co r s
5 co r s
6 rs
7 or s
8 co r s
co e s
s
CO AD
CO AD
c e
c s
c s
co s
c s
c s
co s
s
ĨǁƌŝƚĞ;Ϳ
e
e
e
e
e
re
A
A
CO
CO
3 ore
4 ore
5 ore
6 re
7 ore
8 ore
re
SU
SU
2 cor
ĨĐůŽƐĞ;Ϳ
2 co
AT
AT
ŽĚLJ
o
ƉŝůŽŐƵĞ
c
c
1
1
(a) Kernel execution. (b) Workload throughput. (c) CPU utilization. (d) Latency breakdown. (e) Energy breakdown.
Figure 3: Performance bottleneck analysis of low-power heterogeneous computing.
WĂƌĂůůĞů džĞĐƵƚŝŽŶ ĚĚƌĞƐƐ WĂƌĂůůĞů džĞĐƵƚŝŽŶ
ŵĂŶĂŐĞŵĞŶƚ ĚĚƌĞƐƐ
ŵĂŶĂŐĞŵĞŶƚ
&W'
&W'put them to an improper use. This approach may also intro- ƉƉϬ ŬϬ Ŭϭ Ϭ ŬϬ Ŭϭ ŬϬ
>tW/
duce another type of data movement overheads as frequent ƉƉϮ ŬϮ Ŭϯ ϭ Ŭϭ
ƌƌŝǀĞ Ϯ ŬϮ Ŭϯ ŬϮ
communications are required to use diverse FlashAbacus re- dϬ dŝŵĞ ϯ Ŭϯ
sources from outside. Therefore, our accelerator internally dϬ dϭ dϮ dϯ dϰ dϱ dϲ dϬ dϭ dϮ dϯ dϰ dϱ dϲ
governs all the kernels based on two different scheduling ŬϬ >d Ez >d Ez
Ŭϭ >d Ez >d ^s
models: i) inter-kernel execution and ii) intra-kernel exe- ŬϮ >d Ez >d Ez
Ŭϯ >d Ez >d Ez ^s
cution. In general, in inter-kernel executions, each LWP is
dedicated to execute a specific kernel that performs data pro- D FRQILJXUDWLRQ E ,QWHUNHUQHO VWDWLF F ,QWHUNHUQHO G\QDPLF
cessing from the beginning to the end as a single instruction
stream. In contrast, the intra-kernel execution splits a kernel Figure 5: Examples of inter-kernel scheduling.
into multiple code blocks and concurrently executes them
across multiple LWPs based on the input data layout. The
system(s) and OS kernel memory module(s), our Flashvisor
scheduling details will be explained in Section 4.
internally virtualizes the underlying flash backbone to offer
3.3 Fusing Flash into a Multicore System a large-size of byte-addressable storage without any system
software support. The implementation details of flash virtu-
The lack of file and runtime systems introduces several tech- alization will be explained in Section 4.3.
nical challenges to multi-kernel execution, including mem-
ory space management, I/O management, and resource pro-
tection. An easy-to-implement mechanism to address such
4. Implementation Details
issues is to read and write data on flash through a set of Kernel. The kernels are represented by an executable object
customized interfaces that the flash firmware may offer; this [68], referred to as kernel description table. The descrip-
is the typically adopted mechanism in most active SSD ap- tion table, which is a variation of the executable and link-
proaches [33, 59]. Unfortunately, this approach is inadequate able format (ELF) [16], includes an executable that contains
for our low-power accelerator platform. Specifically, as the several types of section information such as the kernel code
instruction streams (kernels) are independent of each other, (.text), data section (.ddr3 arr), heap (.heap), and stack
they cannot dynamically be linked with flash firmware in- (.stack). In our implementation, all the addresses of such
terfaces. Furthermore, for the active SSD approaches, all ex- sections point to the L2 cache of each LWP, except for the
isting user applications must be modified by considering the data section, which is managed by Flashvisor.
flash interfaces, leading to an inflexible execution model. Offload. A user application can have one or more kernels,
Instead of allowing multiple kernels to access the flash which can be offloaded from a host to a designated memory
firmware directly through a set of static firmware interfaces, space of DDR3L through PCIe. The host can write the kernel
we allocate an LWP to govern the memory space of the data description table associated with the target kernel to a PCIe
section of each LWP by considering flash address spaces. base address register (BAR), which is mapped to DDR3L
This component, referred to as Flashvisor, manages the log- by the PCIe controller (cf. Figure 2b).
ical and physical address spaces of the flash backbone by Execution. After the completion of the kernel download(s),
grouping multiple physical pages across different dies and the host issues a PCIe interrupt to the PCIe controller,
channels, and it maps the logical addresses to the memory and then the controller internally forwards the interrupt to
of the data section. Note that all the mapping information Flashvisor. Flashvisor puts the target LWP (which will ex-
is stored in the scratchpad, while the data associated with ecute the kernel) in sleep mode through power/sleep con-
each kernel’s data sections are placed into DDR3L. In ad- troller (PSC) and stores DDR3L address of such a down-
dition, Flashvisor isolates and protects the physical address loaded kernel to a special register, called boot address regis-
space of flash backbone from the execution of multiple ker- ter of the target LWP. Flashvisor then writes an inter-process
nels. Whenever the kernel loaded to a specific LWP requires interrupt register of the target LWP, forcing this LWP to
accessing its data section, it can inform Flashvisor about the jump to the address written in the boot address register.
logical address space where the target data exist by passing Lastly, Flashvisor pulls the target LWP out of the sleep mode
a message to Flashvisor. Flashvisor then checks a permis- through PSC. Once this revocation process is completed, the
sion of such accesses and translates them to physical flash target LWP begins to load and execute the specified kernel.
address. Lastly, Flashvisor issues the requests to the under- Thus, Flashvisor can decide the order of kernel executions
ling flash backbone, and the FPGA controllers bring the data within an LWP across all LWPs.
to DDR3L. In this flash virtualization, most time-consuming
tasks such as garbage collection or memory dump are pe- 4.1 Inter-kernel Execution
riodically performed by a different LWP, which can ad- Static inter-kernel scheduling. The simplest method to ex-
dress potential overheads brought by the flash management ecute heterogeneous kernels across multiple LWPs is to allo-
of Flashvisor (cf. Section 4.3). Note that, in contrast to a cate each incoming kernel statically to a specific LWP based
conventional virtual memory that requires paging over file on the corresponding application number. Figures 5a shows
6&KZũсϬ͘͘ϯ ŬϬ Ŭϭ
ĞLJϬũсͺĨŝĐƚͺϬ >tWϬ
>tWϬ
E&KZ DŝĐƌŽďůŽĐŬϬ ƉƉϬ ŬϬ
ϭ Ϯ Ă ď ϭ Ă
ŬϬ
ϭ Ϯ DŝĐƌŽďůŽĐŬϬ
>tWϭ
&KZŝсϬ͘͘ϯ
>tWϭ ŬϮ Ŭϯ
Ă ď DŝĐƌŽďůŽĐŬϭ
&KZũсϬ͘͘ϯ ƉƉϮ ϭ Ϯ Ă ŬϬ
ŬϬ ϭ Ϯ Ă ď
>tWϮ
>tWϮ
ĞLJŝũс ƌƌŝǀĞdŝŵĞ
>tWϯ
E&KZ >tWϯ dϬ
&KZũсϭ͘͘ϯ
/ŶƉƵƚ͗ D $SSOLFDWLRQFRQILJXUDWLRQ
E&KZ /ŶƉƵƚ͗ ĞdžŵĂƚƌŝdž
E&KZ DŝĐƌŽďůŽĐŬϭ >tWϬ >tWϭ >tWϮ >tWϯ >tWϬ ŬϬ
ϭ ŬϬ ϭ ŬϬ
Ă ŬϬ Ă ŬϬ
ϭ ŬϬĂ ŬϬ
ϭ ŬϬ
Ă >tWϬ ŬϬ
ĞLJŵĂƚƌŝdž Ă Ϯ ŬϬ
ϭ ŬϬ Ă
&KZŝсϬ͘͘ϯ >tWϭ Ϯ ď Ϯ Ϯ ŬϬ
ď
&KZũсϬ͘͘ϯ >tWϭ Ϯ ď ŬϬ ϭ ŬϬ
Ă
ƐĐƌĞĞŶ KƵƚƉƵƚ͗ >tWϮ ŬϮ Ŭϯ >tWϮ ŬϬ
ϭ ŬϬ
Ă Ϯ ŬϬď
ŚnjŵĂƚƌŝdž >tWϯ >tWϯ ŬϬ
ϭ
E&KZ dϬ dϭ dϮ dϯ dϰ dϱ dϲ dϳ dϬ dϭ dϮ dϯ dϰ dϱ dϲ dϳ
E&KZ DŝĐƌŽďůŽĐŬϮ DŝĐƌŽďůŽĐŬϮ ŬϬ >d͘ ^s ŬϬ >d͘ ^s
Ŭϭ >d Ez ^s Ŭϭ >d͘ ^s
D )'7''DOJRULWKP E 7KHLQSXWOD\RXWRI0LFUREORFN ŬϮ >d Ez ŬϮ >d Ez
Ŭϯ >d Ez Ŭϯ >d Ez ^s
E ,QWUDNHUQHO LQRUGHU F ,QWUDNHUQHO RXWRIRUGHU
Figure 6: Microblocks/screens in an NDP-kernel.
Figure 7: Examples of intra-kernel scheduling.
that has two user applications, App0 and App2, each of which
contains two kernels, respectively (e.g., k0/k1 for App0 and
k2/k3 for App2). In this example, a static scheduler assigns which can be executed across different LWPs. For example,
all kernels associated with App0 and App2 to LWP0 and Figure 6a shows the multiple microblocks observed in the
LWP2, respectively. Figure 5b shows the timing diagram of kernel of (FDTD-2D) in Yee’s method [57]. The goal of this
each LWP execution (top) and the corresponding latency of kernel is to obtain the final output matrix, hz, by processing
each kernel (bottom). Once a host issues all the kernels in the input vector, fict . Specifically, in microblock 0 (m0),
App0 and App2, it does not require any further communica- this kernel first converts fict (1D array) to ey (2D array).
tion with the host until all computations are completed. Even The kernel then prepares new ey and ex vectors by calcu-
though this static inter-kernel scheduling is easy to imple- lating ey/hz and ex/hz differentials in microblock 1 (m1).
ment and manage in the multi-kernel execution model, such These temporary vectors are used for getting the final out-
scheduling can unfortunately lead to a poor resource utiliza- put hz at microblock 2 (m2). In m2, the execution codes per
tion due to the imbalance of kernel loads. For example, while (inner loop) iteration generate one element of output vector,
LWP1 and LWP3 in idle, k1 and k3 should be suspended un- hz, at one time. Since there are no risks of write-after-write
til the previously-issued k0 and k2 are completed. or read-after-write in m2, we can split the outer loop of m2
Dynamic inter-kernel scheduling. To address the poor uti- into four screens and allocate them across different LWPs
lization issue behind static scheduling, Flashvisor can dy- for parallel executions. Figure 6b shows the input and out-
namically allocate and distribute different kernels among put vector splitting and their mapping to the data sections of
LWPs. If a new application has arrived, this scheduler as- four LWPs to execute each screen.
signs the corresponding kernels to any available LWPs in a In-order intra-kernel scheduling. This scheduler can sim-
round robin fashion. As shown in Figure 5c, k1 and k3 are ply assign various microblocks in a serial order, and simulta-
allocated to LWP1 and LWP3, and they are executed in par- neously execute the numerous screens within a microblock
allel with k0 and k2. Therefore, the latency of k1 and k3 by distributing them across multiple LWPs. Figure 7b shows
are reduced as compared to the case of the static scheduler, an example of an in-order inter-kernel scheduler with the
by 2 and 3 time units, respectively. Since each LWP informs same scenario as that explained for Figure 5b. As shown
the completion of kernel execution to Flashvisor through the in Figure 7a, k0’s m0 contains two screens ( 1 and 2 ),
hardware queue (cf. Figure 2), Flashvisor can consecutively which are concurrently executed by different LWPs (LWP0
allocate next kernel to the target LWP. Therefore, this dy- and LWP1). This scheduling can reduce 50% of k0’s la-
namic scheduler can fully utilize LWPs if it can secure suffi- tency compared to the static inter-kernel scheduler. Note that
cient kernel execution requests. However, there is still room this scheduling method can shorten the individual latency for
to reduce the latency of each kernel while keeping all LWPs each kernel by incorporating parallelism at the screen-level,
busy, if we can realize finer scheduling of kernels. but it may increase the total execution time if the data size
is not sufficiently large to partition as many screens as the
4.2 Intra-kernel Execution number of available LWPs.
Microblocks and screens. In FlashAbacus, a kernel is com- Out-of-order intra-kernel Scheduling. This scheduler can
posed of multiple groups of code segments, wherein the exe- perform an out of order execution of many screens associ-
cution of each depends on their input/output data. We refer to ated with different microblocks and different kernels. The
such groups as microblocks. While the execution of different main insight behind this method is that, the data depen-
microblocks should be serialized, in several operations, dif- dency only exists among the microblocks within an appli-
ferent parts of the input vector can be processed in parallel. cation’s kernel. Thus, if there any available LWPs are ob-
We call these operations (within a microblock) as screens, served, this scheduler borrows some screens from a different
7DƵůƚŝͲĂƉƉ džĞĐƵƚŝŽŶŚĂŝŶ responding page table entry, which contains the address of
ƉƉϬ ƉƉϭ ƉƉŶ physical page group number. It then divides the translated
ŵŝĐƌŽďůŽĐŬϬ group number by the total page number of each flash pack-
ŵŝĐƌŽďůŽĐŬϬ
ƐĐƌĞĞŶϬ ƐĐƌĞĞŶϭ ƐĐƌĞĞŶϬ ƐĐƌĞĞŶϭ age, which indicates the package index within in a channel,
and the remainder of the division can be the target physical
ŵŝĐƌŽďůŽĐŬϭ
ƐĐƌĞĞŶϬ ƐĐƌĞĞŶϭ ƐĐƌĞĞŶϮ ĚĞƉĞŶĚĂŶĐLJ page number. Flashvisor creates a memory request target-
ŵŝĐƌŽďůŽĐŬŶ ing the underlying flash backbone, and then all the FPGA
ĚĞƉĞŶĚĂŶĐLJ controllers take over such requests. On the other hand, for a
write request, Flashvisor allocates a new page group number
Figure 8: Data structure of multi-app execution chain. by simply increasing the page group number used in a pre-
vious write. In cases where there is no more available page
group number, Flashvisor generates a request to reclaim a
microblock, which exist across different kernel or applica- physical block. Note that, since the time spent to lookup and
tion boundaries, and allocate the available LWPs to execute update the mapping information should not be an overhead
these screens. Similar to the dynamic inter-kernel scheduler, to virtualize the flash, the entire mapping table resides on
this method keeps all LWPs busy, which can maximize pro- scratchpad; to cover 32GB flash backbone with 64KB page
cessor utilization, thereby achieving high throughput. Fur- group (4 channels * 2 planes per die * 8KB page), it only
thermore, the out-of-order execution of this scheduler can requires 2MB for address mapping. Considering other infor-
reduce the latency of each kernel. Figure 7c shows how mation that Flashvisor needs to maintain, a 4MB scratchpad
the out-of-order scheduler can improve system performance, is sufficient for covering the entire address space. Note that,
while maximizing processor utilization. As shown in the fig- since the page table entries associated with each block are
ure, this scheduler pulls the screen 1 of k1’s microblock also stored in the first two pages within the target physical
0 (m0) from time unit (T1) and executes it at T0. This is be- block of flash backbone (practically used for metadata [81]),
cause LWP2 and LWP3 are available even after executing all the persistence of mapping information is guaranteed.
screens of k0’s microblock 0 (m0), 1 and 2 . With a same Protection and access control. While any of the multiple
reason, the scheduler pulls the screen a of k1’s microblock kernels in FlashAbacus can access the flash backbone, there
2 (m2) from T3 and execute it at T1. Similarly, the screen 1 is no direct data path between the FGPA controllers and
of k2’s microblock 1 (m1) can be executed at T1 instead of other LWPs that process the data near flash. That is, all re-
T2. Thus, it can save 2, 4 and 4 time units for k0, k1 and k2, quests related to flash backbone should be taken and con-
respectively (versus the static inter-kernel scheduler). trolled by Flashvisor. An easy way by which Flashvisor can
Note that, in this example, no screen is scheduled before protect the flash backbone is to add permission information
the completion of all the screens along with a previous mi- and the owner’s kernel number for each page to the page
croblock. In FlashAbacus, this rule is managed by multi-app table entry. However, in contrast to the mapping table used
execution chain, which is a list that contains the data depen- for main memory virtualization, the entire mapping infor-
dency information per application. Figure 8 shows the struc- mation of Flashvisor should be written in persistent storage
ture of multi-app execution chain; the root contains multiple and must be periodically updated considering flash I/O ser-
pointers, each indicating a list of nodes. Each node main- vices such as garbage collection. Thus, adding such tempo-
tains a series of screen information per microblock such as rary information to the mapping table increases the complex-
LWP ID and status of the execution. Note that the order ity of our virtualization system, which can degrade overall
of such nodes indicates the data-dependency relationships system performance and shorten the life time of the under-
among the microblocks. lying flash. Instead, Flashvisor uses a range lock for each
flash-mapped data section. This lock mechanism blocks a
4.3 Flash Virtualization request to map a kernel’s data section to flash if its flash
The kernels on all LWPs can map the memory regions of address range overlaps with that of another by considering
DDR3L pointed by their own data sections to the designated the request type. For example, the new data section will
flash backbone addresses. As shown in Figure 9, individual be mapped to flash for reads, and such flash address is be-
kernel can declare such flash-mapped space for each data ing used for writes by another kernel, Flashvisor will block
section (e.g., input vector on DDR3L) by passing a queue the request. Similarly, a request to map for writes will be
message to Flashvisor. That is, the queue message contains blocked if the address of target flash is overlapped with an-
a request type (e.g., read or write), a pointer to the data sec- other data section, which is being used for reads. Flashvisor
tion, and a word-based address of flash backbone. Flashvi- implements this range lock by using red black tree structure
sor then calculates the page group address by dividing the [65]; the start page number of the data section mapping re-
input flash backbone address, with the number of channel. quest is leveraged as a key, and each node is augmented with
If the request type is a read, Flashvisor refers its page map- the last page number of the data section and mapping request
ping table with the page group number, and retrieves the cor- type.
8ϭ tƌŝƚĞ
ϲ ZĞĂĚ
ϭ DĞƐƐĂŐĞ Ϯ DĞƐƐĂŐĞ
Śη WĂŐĞŐƌŽƵƉη ϱ D
ϱ D ϲ WĂŐĞƚĂďůĞƵƉĚĂƚĞ ϰ ZĞĐůĂŝŵďůŽĐŬ
ϯ WĂŐĞƚĂďůĞůŽŽŬƵƉ
WĂŐĞdĂďůĞ ^ĐƌĂƚĐŚ ϰ /ͬK ^ĐƌĂƚĐŚ ^ƚŽƌĞŶŐŝŶĞ
&ůĂƐŚǀŝƐŽƌ
/ŶĚĞdž
ƉĂĚ &ůĂƐŚǀŝƐŽƌ ƉĂĚ
ϯ >ŽĐŬŝŶƋƵŝƌLJ ϱ /ͬK WĂŐĞƚĂďůĞ 'ĂƌďĂŐĞ
ƐŶĂƉƐŚŽƚ ĐŽůůĞĐƚŝŽŶ
Ϯ >ŽĐŬ
ŝŶƋƵŝƌLJ ^ĞĂƌĐŚ ^ƚĂƌƚ
WĂŐĞ &W'
ZĂŶŐĞ &W' ^ƚĂƌƚ ^ƚĂƌƚ ZĂŶŐĞ
&ůĂƐŚ
&ůĂƐŚ
&ůĂƐŚ
WĂŐĞ ƉĂŐĞ ůŽĐŬ
&ůĂƐŚ
&ůĂƐŚ
&ůĂƐŚ
ƉŬŐη ůŽĐŬ
^ƚĂƌƚ ^ƚĂƌƚ ^ƚĂƌƚ ^ƚĂƌƚ
WĂŐĞ WĂŐĞ WĂŐĞ WĂŐĞ
(a) Reads. (b) Writes.
Figure 9: Examples of flash virtualization procedures in FlashAbacus.
Name Description MBLKs Serial Input LD/ST B/KI Heterogeneous workloads
MBLK (MB) ratio(%) 1 2 3 4 5 6 7 8 9 10 11 12 13 14
ATAX Matrix Transpose&Multip. 2 1 640 45.61 68.86 ● ● ● ●
BICG BiCG Sub Kernel 2 1 640 46 72.3 ● ● ● ●
2DCON 2-Dimension Convolution 1 0 640 23.96 35.59 ● ● ● ● ●
MVT Matrix Vector Prod.&Trans. 1 0 640 45.1 72.05 ● ● ● ● ● ● ● ● ●
ADI Direction Implicit solver 3 1 1920 23.96 35.59 ● ● ● ● ● ● ● ● ●
FDTD 2-D Finite Time Domain 3 1 1920 27.27 38.52 ● ● ● ● ● ● ● ●
GESUM Scalar, Vector&Multip. 1 0 640 48.08 72.13 ● ● ● ● ● ● ● ●
SYRK Symmetric rank-k op. 1 0 1280 28.21 5.29 ● ● ● ● ●
3MM 3-Matrix Multiplications 3 1 2560 33.68 2.48 ● ● ● ●
COVAR Covariance Computation 3 1 640 34.33 2.86 ● ● ● ● ●
GEMM Matrix-Multiply 1 0 192 30.77 5.29 ● ● ● ● ● ● ● ●
2MM 2-Matrix Multiplications 2 1 2560 33.33 3.76 ● ● ● ● ● ● ●
SYR2K Symmetric rank-k op. 1 0 1280 30.19 1.85 ● ● ● ●
CORR Correlation Computation 4 1 640 33.04 2.79 ● ● ● ●
Table 2: Important characteristics of our workloads.
Storage management. Flashvisor can perform address the physical block from the beginning of flash address space
translation similar to a log-structured pure page mapping to the end in background. Most garbage collection and wear-
technique [42], which is a key function of flash firmware leveling algorithms that are employed for flash firmware se-
[3]. To manage the underlying flash appropriately, Flashvi- lect victim blocks by considering the number of valid pages
sor also performs metadata journaling [3] and garbage col- and the number of block erases. These approaches increase
lection (including wear-leveling) [10, 11, 38, 60]. However, the accuracy of garbage collection and wear-leveling, since
such activities can be strong design constraints for SSDs as they require extra information to set such parameters and
they should be resilient in all types of power failures, includ- search of the entire address translation information to iden-
ing sudden power outages. Thus, such metadata journaling tify a victim block. Rather than wasting compute cycles to
is performed in the foreground and garbage collection are examine all the information in the page table, Storengine
invoked on demand [10]. In cases where an error is detected simply selects the target victim block from a used block
just before an uncorrectable error correction arises, Flashvi- pool in a round robin fashion and loads the corresponding
sor excludes the corresponding block by remapping it with page table entries from flash backbone. Storengine periodi-
new one. In addition, all the operations of these activities are cally migrates valid pages from the victim block to the new
executed in the lockstep with address translation. block, and return the victim to a free block in idle, which is
However, if these constraints can be relaxed to some ex- a background process similar to preemtable firmware tech-
tent, most overheads (that flash firmware bear brunt of) are niques [38, 40]. Once the victim block is selected, the page
removed from multi-kernel executions. Thus, we assign an- mapping table associated with those two blocks is updated
other LWP to perform this storage management rather than in both the scratchpad and flash. Note that all these activities
data processing. While Flashvisor is responsible for mainly of Storengine can be performed in parallel with the address
address translation and multi-kernel execution scheduling, translation of Flashvisor. Thus, locking the address ranges
this LWP, referred to as Storengine, periodically dumps the that Storengine generates for the snapshot of the scratchpad
scratchpad information to the underlying flash as described (journaling) or the block reclaim is necessary, but such ac-
in the previous subsection. In addition, Storengine reclaims
9tivities overlapped with the kernel executions and address culator, and use our in-house power analyzer to monitor the
translations (are performed in the background). dynamic power of SSD [34, 79].
5.1 Data Processing Throughput
5. Evaluation
Figure 10 demonstrates the overall throughput of all hard-
Accelerators. We built five different heterogeneous comput- ware accelerators that we implemented by executing both
ing options. “SIMD” employs a low-power accelerator that homogeneous workloads and heterogeneous workloads.
executes multiple data-processing instances using a single- Generally speaking, the proposed FlashAbacus approach
instruction multiple-data (SIMD) model implemented in (i.e., IntraO3) outperforms the conventional accelerator
OpenMP [49], and SIMD performs I/Os through a discrete approach SIMD by 127%, on average, across all workloads
high-performance SSD (i.e., Intel NVMe 750 [32]). On tested.
the other hand, “InterSt”, “InterDy”, “IntraIo”, and Homogeneous workloads. Figure 10a shows the over-
“IntraO3” employ the static inter-kernel, dynamic inter- all throughput evaluated from homogeneous workloads. In
kernel, in-order intra-kernel, and out-of-order intra-kernel this evaluation, we generate 6 instances from each kernel.
schedulers of FlashAbacus, respectively. In these acceler- Based on the computation complexity (B/KI), we catego-
ated systems, we configure the host with Xeon CPU [31] rize all workloads into two groups: i) data-intensive and
and 32GB DDR4 DRAM [64]. ii) computing-intensive. As shown in the figure, all our
Benchmarks. We also implemented 14 real benchmarks that FlashAbacus approaches outperform SIMD for data-intensive
stem from Polybench [57] on all our accelerated systems2 . workloads by 144%, on average. This is because, although
To evaluate the impact of serial instructions in the multi-core all the accelerators have equal same computing powers,
platform, we rephrase partial instructions of tested bench- SIMD cannot process data before the data are brought by the
marks to follow a manner of serial execution. The configu- host system. However, InterSt has poorer performance,
ration details and descriptions for each application are ex- compared to InterDy and IntraO3, by 53% and 52%, re-
plained in Table 2. In this table, we explain how many mi- spectively. The reason is that InterSt is limited to spread-
croblocks exist in an application (denoted by MBLKs). Se- ing homogeneous workloads across multiple workers, due to
rial MBLK refers to the number of microblocks that have no its static scheduler configuration. Unlike InterSt, IntraIo
screens; these microblocks are to be executed in serial. The enjoys the benefits of parallelism, by partitioning kernels
table also provides several workload analyses, such as the into many screens and assigning them to multiple LWPs.
input data size of an instance (Input), ratio of load/store in- However, IntraIo cannot fully utilize multiple LWPs if
structions to the total number of instructions (LD/ST ratio), there is a microblock that has no code segments and that can-
and computation complexity in terms of data volumes to pro- not be concurrently executed (i.e., serial MBLK). Compared
cess per thousand instructions (B/KI). To evaluate the bene- to IntraIo, IntraO3 successfully overcomes the limits of
fits of different accelerated systems, we also created 14 het- serial MBLK by borrowing multiple screens from different
erogeneous workloads by mixing six applications. All these microblocks, thereby improving the performance by 62%,
configurations are presented at the right side of the table, on average. For homogeneous workloads, InterDy achieves
where the symbol • indicates the correspondent applications, the best performance, because all the kernels are equally as-
which are included for such heterogeneous workloads. signed to a single LWP and are simultaneously completed.
Profile methods. To analyze the execution time breakdown While IntraO3 can dynamically schedule microblocks to
of our accelerated systems, we instrument timestamp an- LWPs for a better load balance, IPC overheads (between
notations in both host-side and accelerator-side application Flashvisor and workers) and scheduling latency of out-of-
source codes by using a set of real-time clock registers and order execution degrade IntraO3’s performance by 2%, on
interfaces. The annotations for each scheduling point, data average, compared to InterDy.
transfer time, and execution period enable us to capture the Heterogeneous workloads. Figure 10b shows the overall
details of CPU active cycles and accelerator execution cy- system bandwidths of different accelerated systems under
cles at runtime. On the other hand, we use a blktrace analysis heterogeneous workloads. For each execution, we gener-
[6] to measure device-level SSD performance by removing ate 24 instances, four from a single kernel. SIMD exhibits
performance interference potentially brought by any mod- 3.5 MB/s system bandwidth across all heterogeneous work-
ules in storage stack. We leverage Intel power gadget tool to loads, on average, which is 98% worse than that of data-
collect the host CPU energy and host DRAM energy [43]. intensive workloads. This is because, heterogeneous work-
Lastly, we estimate the energy consumption of FlashAbacus loads contain computing-intensive kernels, which involve
hardware platform based on TI internal platform power cal- much longer latency in processing data than storage ac-
2 In this implementation, we leverage a set of TI code generation tools
cesses. Although the overheads imposed by data movement
[66, 67, 69] to develop and compile the kernels of our FlashAbacus. Note
are not significant in computing-intensive workloads, the
that the memory protection and access control are regulated by the TI’s limited scheduling flexibility of SIMD degrades performance
tools, which may be limited for other types of acceleration hardware. by 42% and 50%, compared to InterDy and IntraO3, re-
10SIMD InterSt IntraIo InterDy IntraO3 SIMD InterSt IntraIo InterDy IntraO3
Throughput (MB/s)
Throughput (MB/s)
1200 data-intensive compute-intensive 24
600 16
8
20 4
10 2
0 I 0
AX CG ON VT UM AD DTDYRK3MMVAREMM2MMR2KORR 1 2 3 4 5 6 7 8 9 0 1 2 3 4
AT BI2DC M ES F S CO G SY C MX MX MX MX MX MX MX MX MXMX1MX1MX1MX1MX1
G
(a) Performance analysis for homogeneous workloads. (b) Performance analysis for heterogeneous workloads.
Figure 10: Performance analysis (computation throughput normalized to SIMD).
MAX AVG MIN MAX AVG MIN
Norm. latency
Norm. latency
2 M:SIMD S:InterSt 4.3 S ID 7.8 8.4 2 M:SIMD S:InterSt I:IntraIo
I:IntraIo D:InterDy 2.6 M O 4.5 4.9 D:InterDy O:IntraO3 MS I D 3.7 3.5
O:IntraO3 O
1 1
0 0
X G ON T UM I D K M AR M M 2 K R 1 2 3 4 5 6 7 8 9 0 1 2 3 4
ATA BIC 2DC MVGES AD FDT SYR 3M COV GEM 2M SYR COR MX MX MX MX MX MX MX MX MX MX1MX1 MX1MX1MX1
(a) The latency analysis for homogeneous workloads. (b) The latency analysis for heterogeneous workloads.
Figure 11: Latency analysis (normalized to SIMD).
# of completed kernels
IntraIo
# of completed kernels
SIMD InterSt IntraIo SIMD InterSt 5.2 Execution Time Analysis
InterDy IntraO3 InterDy IntraO3
6 6 Homogeneous workloads. Figure 11a analyzes latency of
5 5
4 4 the accelerated systems (with homogeneous workloads). For
3 3 data-intensive workloads (i.e., ATAX, BICG, 2DCONV, and
2 2 MVT), average latency, maximum latency, and minimum la-
1 1
0 0
tency of SIMD are 39%, 87%, and 113% longer than those
0.0 0.5 1.0 1.5 2.0 0 2 4 6 200 400 of FlashAbacus approaches, respectively. This is because,
Completion Time (s) Completion Time (s)
SIMD consumes extra latency to transfer data through dif-
(a) Latency analysis (ATAX). (b) Latency analysis (MX1).
ferent I/O interfaces and redundantly copy I/Os across dif-
Figure 12: CDF analysis for workload latency. ferent software stacks. InterSt and InterDy exhibit simi-
lar minimum latency. However, InterDy has 57% and 68%
shorter average latency and maximum latency, respectively,
compared to InterSt. This is because, InterSt can dy-
namically schedule multiple kernels across different work-
spectively. In contrast to the poor performance observed in ers in parallel. The minimum latency of inter-kernel sched-
homogeneous workloads, InterSt outperforms IntraIo ulers (InterSt and InterDy) is 61% longer than that of
by 34% in workloads MX2,3,4,6,7,9 and 13. This is because intra-kernel schedulers (IntraIo and IntraO3), because
SIMD can statically map 6 different kernels to 6 workers. intra-kernel schedulers can shorten single kernel execution
However, since different kernels have various data process- by leveraging multiple cores to execute microblocks in par-
ing speeds, unbalanced loads across workers degrade per- allel.
formance by 51%, compared to IntraIo, for workloads Heterogeneous workloads. Figure 11b shows the same la-
MX1,5,8,10,11 and 12. Unlike InterSt, InterDy monitors tency analysis, but with heterogeneous workloads. All het-
the status of workers and dynamically assigns a kernel to erogeneous workloads have more computing parts than the
a free worker to achieve better load balance. Consequently, homogeneous ones, data movement overheads are hidden
InterDy exhibits 177% better performance than InterSt. behind their computation somewhat. Consequently, SIMD
Compared to the best performance reported by the previous exhibits kernel execution latency similar to that of IntraIo.
homogeneous workload evaluations, performance degrada- Meanwhile, InterSt exhibits a shorter maximum latency
tion of InterDy in these workloads is caused by a stagger for workloads MX4,5,8 and 14, compared to IntraIo. This
kernel, which exhibits much longer latency than other ker- is because, these workloads contain multiple serial mi-
nels. In contrast, IntraO3 shortens the latency of the stag- croblocks that stagger kernel execution. Further InterSt
ger kernel by executing it across multiple LWPs in parallel. has the longest average latency among all tested approaches
As a result, IntraO3 outperforms InterDy by 15%. due to inflexible scheduling strategy. Since IntraO3 can
11Energy breakdown
Energy breakdown
data movement computation storage access data movement computation storage access
1.5
1.2 M:SIMD S:InterSt I:IntraIo D:InterDy O:IntraO3 1.2 M:SIMD S:InterSt I:IntraIo D:InterDy O:IntraO3
S ID 0.9
0.9 M O
0.6 I
0.6 S D
O
0.3 M
0.3
0.0 I
AX GCONMVTSUM AD DTDYRK3MMVAREMM2MM R2KORR 0.0
AT BIC 1 2 3 4 5 6 7 8 9 0 1 2 3 4
2D GE F S CO G SY C MX MX MX MX MX MX MX MX MX MX1MX1MX1MX1MX1
(a) Energy breakdown for homogeneous workloads. (b) Energy breakdown for heterogeneous workloads.
Figure 13: Analysis for energy decomposition (all results are normalized to SIMD).
Processor utilization(%)
Processor utilization(%)
SIMD InterSt IntraIo InterDy IntraO3 SIMD InterSt IntraIo InterDy IntraO3
120 data-intensive compute-intensive
100 100
80 80
60 60
40 40
20 20
0 0
AX G N VT M DI TD RK M AR M M 2K R
AT BIC2DCO MGESU A FD SY 3MCOV GEM 2MSYR COR 1 2 3 4 5 6 7 8 9 0 1 2 3 4
MX MX MX MX MX MX MX MX MX MX1 MX1 MX1 MX1 MX1
(a) The ratio of LWP utilization for homogeneous workloads. (b) The ratio of LWP utilization for heterogeneous workloads.
Figure 14: Analysis of processor utilization.
split kernels into multiple screens and achieve a higher level always busy for their entire execution, leading to ineffi-
of parallelism with a finer-granule scheduling, it performs cient energy consumption behavior. However, IntraIo,
better than InterDy in terms of the average and maximum InterDy, and IntraO3 reduce the overall energy con-
latency by 10% and 19%, respectively. sumption by 47%, 22%, and 2%, respectively, compared
CDF analysis. Figures 12a and 12b depict the execution to InterSt. This is because IntraO3 can exploit the mas-
details of workload ATAX and MX1, each representing ho- sive parallelism (among all workers), saving energy, wasted
mogeneous and heterogeneous workloads, respectively. As by the idle workers (observed by InterSt).
shown in Figure 12a, InterDy takes more time to com-
plete the first kernel compared to IntraIo and IntraO3, be-
cause InterDy only allocates single LWP to process the first
kernel. For all the six kernels, InterO3 also completes the
operation earlier than or about the same time as InterDy. 5.4 Processor Utilizations
More benefits can be achieved by heterogeneous execution.
As shown in Figure 12b, for the first four data-intensive Figures 14a and 14b show an analysis of LWP utilizations
kernels, SIMD exhibits much longer latency compared to obtained by executing homogeneous and heterogeneous
FlashAbacus, due to the system overheads. While SIMD out- workloads. In this evaluation, LWP utilization is calculated
performs InterSt for the last two computation-intensive by dividing the LWP’s actual execution time (average) by
kernels, IntraO3 outperforms SIMD by 42%, on average. the latency to execute entire application(s) in each work-
load. For data-intensive ones, most of LWP executions are
stalled due to the long latency of storage accesses, which
5.3 Energy Evaluation make SIMD’s LWP utilizations lower than InterO3 by 23%,
In Figure 13, the energy of all five tested accelerated sys- on average. On the other hand, InterDy keeps all proces-
tems is decomposed into the data movement, computation sors busy by 98% of the total execution time, which leads
and storage access parts. Overall, IntraO3 consumes aver- the highest LWP utilization among all accelerated systems
age energy less than SIMD by 78.4% for all the workloads we tested. IntraO3 schedules microblocks in out-of-order
that we tested. Specifically, all FlashAbacus approaches ex- manner and only a small piece of code segment is executed
hibit better energy efficiency than SIMD for data-intensive each time, which makes its LWP utilization slightly worse
workloads (cf. Figure 13a). This is because, the host re- than InterDy ones (less than 1%) for the execution of ho-
quires periodic transfer of data between the SSD and accel- mogeneous kernels. In contrast, for heterogeneous kernel ex-
erator, consuming 71% of the total energy of SIMD. Note ecutions, IntraO3 achieves processor utilization over 94%,
that, InterSt consumes 28% more energy than SIMD for on average, 15% better than that of InterDy. Since LWP
computing-intensive workloads, GEMM, 2MM, and SYR2K. utilization is strongly connected to the degree of parallelism
Even though InterSt cannot fully activate all workers due and computing bandwidth, we can conclude that IntraO3
to its inflexibility, it must keep Flashvisor and Storengine outperforms all the accelerated systems.
12Functional unit utilization
405 5.6 Extended Evaluation on Real-world Applications
SIMD IntraO3
324
243 parallel parallel Application selection. In this section, we select five rep-
162 storage storage resentative data-intensive workloads coming from graph
access serial serial access
81 complete complete benchmarks [12] and bigdata benchmarks [24] in order to
00 better understand system behaviors with real applications: i)
0 1000 2000 3000 4000 5000 6000 7000 8000
Time (us) K-nearest neighbor (nn), ii) graph traversal (bfs), iii) DNA
(a) The function unit utilization of heterogeneous workloads. sequence searching (nw), iv) grid traversal (path) and v)
mapreduce wordcount (wc)).
60 SIMD IntraO3 Performance. Figure 16a illustrates data processing through-
Power (W)
50
40
storage
put of our FlashAbacus by executing the graph/bigdata
30 access compute complete compute complete
20
benchmarks. One can observe from this figure that the aver-
10 age throughput of FlashAbacus approaches (i.e., IntraIo,
0 InterDy and IntraO3) outperform SIMD by 2.1x, 3.4x and
0 1000 2000 3000 4000 5000 6000 7000 8000
Time (us) 3.4x, respectively, for all data-intensive workloads that we
(b) Power analysis of heterogeneous workloads. tested. Even though SIMD can fully utilize all LWPs, and it is
expected to bring an excellent throughput for the workloads
Figure 15: Resource utilization and power analysis.
that have no serialized faction of microblocks (i.e., nw and
path), SIMD unfortunately exhibits poor performance than
other FlashAbacus approaches. This performance degrada-
SIMD InterSt data movement
computation tion is observed because LWPs are frequently stalled and
IntraIo InterDy
IntraO3 storage access bear brunt of comparably long latency imposed by moving
Energy breakdown
the target data between its accelerator and external SSD over
Throughput (MB/s)
1200 1.2 M:SIMD S:InterSt I:IntraIo
D:InterDy O:IntraO3 multiple interface and software intervention boundaries.
900 0.9 Similarly, while SIMD’s computation capability is power-
600 0.6 S ID ful much more than InterSt’s one, which requires to ex-
M O ecute multiple kernels in a serial order, InterSt’s average
300 0.3
throughput is 120% and 131% better than SIMD’s perfor-
0 0.0 s mance for nw and path, respectively.
bfs wc nn nwpath bf wc nn nw path
Energy. We also decompose the total energy of the sys-
(a) Performance analysis. (b) Energy breakdown.
tem, which is used for each data-intensive application, into
Figure 16: Analysis of graph/big data applications. i) data movement, ii) computation, and iii) storage access;
each item represents the host energy consumed for transfer-
ring data between the accelerator and SSD, the actual energy
used by accelerator to compute, and the energy involved in
5.5 Dynamics in Data Processing and Power
serving I/O requests, respectively. The results of this energy
Figures 15a and 15b illustrate the time series analysis re- breakdown analysis is illustrated by Figure 16b. One can ob-
sults for utilizing functional units and power usage of work- serve from this figure that InterSt, IntraI0, InterDy and
ers, respectively. In this evaluation, we compare the results IntraO3 can save the total average energy for processing
of IntraO3 with those for SIMD to understand in detail the all data by 74%, 83%, 88% and 88%, compared to SIMD, re-
runtime behavior of the proposed system. As shown in Fig- spectively. One of the reasons behind of this energy saving is
ure 15a, IntraO3 achieves shorter execution latency and that the cost of data transfers account for 79% of the total en-
better function unit utilization than SIMD. This is because, ergy cost in SIMD, while FlashAbacus eliminates the energy,
IntraO3 can perform parallel execution of multiple serial wasted in moving the data between the accelerator and SSD.
microblocks in multiple cores, which increases the function We observe that the computation efficiency of InterDy and
unit utilization. In addition, due to the shorter storage ac- IntraO3 also is better than that SIMD for even the workloads
cess, IntraO3 can complete the execution 3600 us earlier that have serial microblocks (i.e., bfs and nn). This is be-
than SIMD. On the other hand, IntraO3 exhibits much lower cause OpenMP that SIMD use should serialize the executions
power consumption than SIMD during storage access (Figure of serial microblocks, whereas Flashvisor of the proposed
15b). This is because SIMD requires the assistance of host FlashAbacus coordinates such serial microblocks from dif-
CPU and main memory in transferring data between the SSD ferent kernels and schedule them across different LWPs in
and accelerator, which in turn consumes 3.3x more power parallel.
than IntraO3. Interestingly, the pure computation power of
IntraO3 is 21% higher than SIMD, because IntraO3 en-
ables more active function units to perform data processing,
thus taking more dynamic power.
136. Discussion and Related Work posed by different physical interface boundaries, and unfor-
Platform selection. The multicore-based PCIe platform we tunately, their solutions highly depends on the target sys-
select [29] integrates a terabit-bandwidth crossbar network tem’s software and software environment.
for multi-core communication, which potentially make the In-storage processing. Prior studies [2, 13, 22, 41, 72] pro-
platform a scale-out accelerator system (by adding up more posed to eliminate the data movement overheads by lever-
LWPs into the network). In addition, considering a low aging the existing controller to execute a kernel in storage.
power budget (5W ∼20W), the platform we implemented However, these in-storage process approaches are limited to
FlashAbacuse has been considered as one of the most en- a single core execution and their computation flexibility is
ergy efficient (and fast) accelerator in processing a massive strictly limited by the APIs built at the design stage. [33] in-
set of data [25]. Specifically, it provides 358 Gops/s with tegrates SSD controller and data processing engine within a
25.6 Gops/s/W power efficiency, while other low-power single FPGA, which supports parallel data access and host
multiprocessors such as a many-core system [54], a mobile task processing. While this FPGA-based approach is well
GPU [51, 52], a FPGA-based accelerator [77] are available optimized for a specific application, porting the current host
to offer 10.5, 256 and 11.5 Gops/s with 2.6, 25.6 and 0.3 programs to RTL design can bring huge burdens to program-
Gops/s/W power efficiency, respectively. mers and still inflexible to adopt many different types of
Limits of this study. While the backend storage complex of data-intensive applications at runtime. FlashAbacus governs
FlashAbacus is designed as a self-existent module, we be- the storage within an accelerator and directly executes mul-
lieve that it is difficult to replace with conventional off-the- tiple general applications across multiple light-weight pro-
shelf SSD solutions. This is because our solution builds flash cessors without any limit imposed by the storage firmware
firmware solution from scratch, which is especially cus- or APIs.
tomized for flash virtualization. We select a high-end FPGA Open-channel or cross-layer optimization. Many propos-
for the flash transaction controls as research purpose, but the als also paid attention on cross-layer optimizations or open-
actual performance of flash management we believe cannot channel SSD approaches that migrates firmware from the
be equal to or better than microcoded ASIC-based flash con- device to kernel or partitions some functionalities of flash
troller due to the limit of low frequency of FPGA. Another controller and implement it into OS [7, 26, 36, 39, 46, 53].
limit behind this work is that we are unfortunately not able While all these studies focused on managing the underlying
to implement FlashAbacus in different accelerators such as flash memory and SSD efficiently, they have a lack of con-
GPGPU due to access limits for such hardware platforms siderations on putting data processing and storage together
and a lack of supports for code generations. However, we in an hardware platform. In addition, since these proposals
believe that the proposed self-governing design (e.g., flash cannot be applied into an single hardware accelerator, which
integration and multi-kernel execution) can be applied to has no storage stack, but requires byte-addressability to ac-
any type of hardware accelerators if vendor supports or open cess the underlying storage complex.
their IPs, which will outperform many accelerators that re-
quire employing an external storage. 7. Conclusion
Hardware/software integration. To address the overheads In this work, we combined flash with lightweight multi-core
of data movements, several hardware approaches tried to di- accelerators to carry out energy-efficient data processing.
rectly reduce long datapath that sit between different com- Our proposed accelerator can offload various kernels in an
puting devices. For example, [4, 20, 61] forward the tar- executable form and process parallel data. Our accelerator
get data directly among different PCIe devices without a can improve performance by 127%, while requiring 78%
CPU intervention. System-on-chip approachs that tightly in- less energy, compared to a multi-core accelerator that needs
tegrate CPUs and accelerators over shared memory [17, 75], external storage accesses for data-processing.
which can reduce the data transfers between. However, these
hardware approaches cannot completely address the over- 8. Acknowledgement
heads imposed by discrete software stacks. Since SSDs are
This research is mainly supported by NRF 2016R1C1B2015312
block devices, which can be shared by many other user ap-
and MemRay grant (2015-11- 1731). This work is also sup-
plications, all the SSD accesses should be controlled by the
ported in part by, DOE DEAC02-05CH 11231, IITP-2017-
storage stack appropriately. Otherwise, the system cannot
2017-0-01015, and NRF2015M3C4A7065645. The authors
guarantee the durability of storage, and simultaneous data
thank Gilles Muller for shepherding this paper. We also
access without an access control of storage stack can make
thank MemRay Corporation and Texas Instruments for their
data inconsistent, in turn leading to a system crash. In con-
research sample donation and technical support. Myoungsoo
trast, a few previous studies explored reducing the overheads
jung is the corresponding author.
of moving data between an accelerator and a storage by op-
timizing the software stack of the two devices [63, 73, 80].
References
However, these studies cannot remove all limitations im-
[1] Nvidia gp100 architecture whitepaper. White paper, 2016.
14You can also read
