Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring Kia Rahmani Kartik Nagar Purdue University, USA IIT Madras, India rahmank@purdue.edu nagark@cse.iitm.ac.in Benjamin Delaware Suresh Jagannathan Purdue University, USA Purdue University, USA bendy@purdue.edu suresh@cs.purdue.edu Abstract Keywords: Schema Refactoring, Serializability, Weak Con- Serializability is a well-understood concurrency control mech- sistency, Weak Isolation anism that eases reasoning about highly-concurrent database ACM Reference Format: programs. Unfortunately, enforcing serializability has a high Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagan- performance cost, especially on geographically distributed nathan. 2021. Repairing Serializability Bugs in Distributed Data- database clusters. Consequently, many databases allow pro- base Programs via Automated Schema Refactoring. In Proceedings grammers to choose when a transaction must be executed of the 42nd ACM SIGPLAN International Conference on Program- under serializability, with the expectation that transactions ming Language Design and Implementation (PLDI ’21), June 20ś would only be so marked when necessary to avoid serious 25, 2021, Virtual, Canada. ACM, New York, NY, USA, 16 pages. concurrency bugs. However, this is a significant burden to https://doi.org/10.1145/3453483.3454028 impose on developers, requiring them to (a) reason about subtle concurrent interactions among potentially interfering transactions, (b) determine when such interactions would 1 Introduction violate desired invariants, and (c) then identify the minimum Programs that concurrently access shared data are ubiqui- number of transactions whose executions should be serial- tous: bank accounts, shopping carts, inventories, and social ized to prevent these violations. To mitigate this burden, this media applications all rely on a shared database to store paper presents a sound and fully automated schema refac- information. For performance and fault tolerance reasons, toring procedure that transforms a program’s data layout the underlying databases that manage state in these appli- ś rather than its concurrency control logic ś to eliminate cations are often replicated and distributed across multiple, statically identified concurrency bugs, allowing more trans- geographically distant locations [34, 36, 48, 51]. Writing pro- actions to be safely executed under weaker and more perfor- grams which interact with such databases is notoriously mant database guarantees. Experimental results over a range difficult, because the programmer has to consider an expo- of realistic database benchmarks indicate that our approach nential space of possible interleavings of database operations is highly effective in eliminating concurrency bugs, with in order to ensure that a client program behaves correctly. safe refactored programs showing an average of 120% higher One approach to simplifying this task is to assume that sets throughput and 45% lower latency compared to a serialized of operations, or transactions, executed by the program are baseline. serializable [40], i.e. that the state of the database is always CCS Concepts: · Software and its engineering → Sys- consistent with some sequential ordering of those transac- tem modeling languages; Application specific development tions. One way to achieve this is to rely on the underlying environments; · Computing methodologies → Distributed database system to seamlessly enforce this property. Unfor- computing methodologies. tunately, such a strategy typically comes at a considerable performance cost. This cost is particularly significant for distributed databases, where the system must rely on expen- sive coordination mechanisms between different replicas, in effect limiting when a transaction can see the effects of another in a way that is consistent with a serializable execu- This work is licensed under a Creative Commons Attribution International 4.0 License. tion [5]. This cost is so high that developers default to weaker PLDI ’21, June 20ś25, 2021, Virtual, Canada consistency guarantees, using careful design and testing to © 2021 Copyright held by the owner/author(s). ensure correctness, only relying on the underlying system ACM ISBN 978-1-4503-8391-2/21/06. to enforce serializable transactions when serious bugs are https://doi.org/10.1145/3453483.3454028 discovered [27, 37, 45, 50]. 32
PLDI ’21, June 20ś25, 2021, Virtual, Canada Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannathan Uncovering such bugs is a delicate and highly error-prone 1. We observe that serializability violations in database task even in centralized environments: in one recent study, programs can be eliminated by changing the schema Warszawski and Bailis [56] examined 12 popular E-Commerce of the underlying database and the client programs applications used by over two million well-known websites in order to eliminate problematic accesses to shared and discovered 22 security vulnerabilities and invariant vi- database state. olations that were directly attributable to non-serializable 2. Using this observation, we develop an automated refac- transactions. To help developers identify such bugs, the com- toring algorithm that iteratively repairs statically iden- munity has developed multiple program analyses that report tified serializability anomalies in distributed database potential serializability anomalies [12, 13, 27, 31, 39]. Auto- clients. We show this algorithm both preserves the se- matically repairing these anomalies, however, has remained mantics of the original program and eliminates many a challenging open problem: in many cases full application identified serializability anomalies. safety is only achievable by relying on the system to enforce 3. We develop a tool, Atropos1 , implementing these strong consistency of all operations. Such an approach re- ideas, and demonstrate its ability to reduce the num- sults in developers either having to sacrifice performance ber of serializability anomalies in a corpus of standard for the sake of correctness, or conceding to operate within benchmarks with minimal performance impact over a potentially restricted ecosystem with specialized services the original program, but with substantially stronger and APIs [4] architectured with strong consistency in mind. safety guarantees. In this paper, we propose a novel language-centric ap- The remainder of the paper is structured as follows. The proach to resolving concurrency bugs that arise in these dis- next section presents an overview of our approach. Section 3 tributed environments. Our solution is to alter the schema, defines our programming model and formalizes the notion or data layout, of the data maintained by the database, rather of concurrency bugs. Section 4 provides a formal treatment than the consistency levels of the transactions that access of our schema refactoring strategy. Sections 5 and 6 describe that data. Our key insight is that it is possible to modify our repair algorithm and its implementation, respectively. shared state to remove opportunities for transactions to wit- Section 7 describes our experimental evaluation. Related ness changes that are inconsistent with serializable execu- work and conclusions are given in Section 8 and Section 9. tions. We, therefore, investigate automated schema trans- formations that change how client programs access data to 2 Overview ensure the absence of concurrency bugs, in contrast to using expensive coordination mechanisms to limit when transac- To illustrate our approach, consider an online course man- tions can concurrently access the database. agement program that uses a database to manage a list of For example, to prevent transactions from observing non- course offerings and registered students. Figure 1 presents a atomic updates to different rows in different tables, we can simplified code snippet implementing such a program. The fuse the offending fields into a single row in a single table database consists of three tables, maintaining information whose updates are guaranteed to be atomic under any consis- regarding courses, students, and their email addresses. The tency guarantee. Similarly, consecutive reads and writes on a STUDENT table maintains a reference to a student’s email row can be refactored into łfunctionalž inserts into a new ta- entry in schema EMAIL (via secondary key st_em_id) and a ble, which removes the race condition between concurrently reference to a course entry in table COURSE (via secondary running instances of the program. By changing the schema key st_co_id) that the student has registered for. A student’s (and altering how transactions access data accordingly), with- registration status is stored in field st_reg. Each entry in out altering a transaction’s atomicity and isolation levels, table COURSE also stores information about the availability we can make clients of distributed databases safer without of a course and the number of enrolled students. sacrificing performance. In our experimental evaluation, we The program includes three sets of database operations or were able to fix on average 74% of all identified serializabil- transactions. Transaction getSt, given a student’s id, first re- ity anomalies with only a minimal impact (less than 3% on trieves all information for that student (S1). It then performs average) on performance in an environment that provides two queries, (S2 and S3), on the other tables to retrieve their only weak eventually consistent guarantees [14]. For the email address and course availability. Transaction setSt remaining 26% of anomalies that were not eliminated by our takes a student’s id and updates their name and email ad- refactoring approach, simply marking the offending trans- dress. It includes a query (S4) and an update (U1) to table actions as serializable yields a provably safe program that STUDENT and an update to the EMAIL table (U2). Finally, trans- nonetheless improves the throughput (resp. latency) of its action regSt registers a student in a course. It consists of fully serialized counterpart by 120% (resp. 45%) on average. an update to the student’s entry (U3), a query to COURSE to This paper makes the following contributions: determine the number of students enrolled in the course they 1 https://github.com/Kiarahmani/AtroposTool 33
Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI ’21, June 20ś25, 2021, Virtual, Canada STUDENT st_id st_name st_em_id st_co_id st_reg concurrent execution of instances of getSt and regSt. Here, COURSE co_id co_avail co_st_cnt EMAIL em_id em_addr (S1) witnesses the effect of (U3) observing that the student is registered, but (S3) sees that the course is unavailable, since 1 getSt ( id ) : 2 x := select * from STUDENT where st_id = id // S1 it does not witness the effect of (U4). This is an instance 3 4 y := select em_addr from EMAIL where em_id = x . st_em_id // S2 z := select co_aval from COURSE where co_id = x . st_co_id // S3 of a dirty-read anomaly. Lastly, the execution on the right shows two instances of regSt that attempt to increment the 1 setSt ( id , name , email ) : 2 x := select st_em_id from STUDENT where st_id = id // S4 number of students in a course. This undesirable behavior, 3 update STUDENT set st_name = name where st_id = id // U1 known as a lost update, leaves the database in a state incon- 4 update EMAIL set em_addr = emal where em_id = x . st_em_id // U2 sistent with any sequential execution of the two transaction 1 regSt ( id , course ) : instances. All of these anomalies can arise if the strong atom- 2 update STUDENT set st_co_id = course , st_reg = true 3 where st_id = id // U3 icity and isolation guarantees afforded by serializability are 4 x := select co_st_cnt from COURSE where co_id = course // S5 5 update COURSE set co_st_cnt = x . co_st_cnt +1 , weakened. 6 co_avail = true where co_id = course // U4 getSt(1): setSt(1,A,a@b): getSt(1): regSt(1,101): regSt(2,101): regSt(1,101): S1 U3 S5 U1 S1 S5 Figure 1. Database schemas and code snippets from an on- U4 U2 S3 U4 line course management program S2 U4 Figure 2. Serializability Anomalies wish to register for (S5), and an update to that course’s avail- ability (U4) indicating that it is available now that a student Several recent proposals attempt to identify such unde- has registered for it. sirable behaviors in programs using a variety of static or The desired semantics of this program is these transactions dynamic program analysis and monitoring techniques [12, should be performed atomically and in isolation. Atomicity 13, 39, 56]. Given potential serializability violations, the stan- guarantees that a transaction never observes intermediate dard solution is to strengthen the atomicity and isolation updates of another transaction. Isolation guarantees that a requirements on the offending transactions to eliminate the transaction never observes changes to the database by other undesirable behaviour, at the cost of increased synchroniza- committed transactions once it begins executing. Taken to- tion overhead or reduced availability [7, 27, 50]. gether, these properties ensure that all executions of this pro- gram are serializable, yielding behavior that corresponds to Atropos. Are developers obligated to sacrifice concur- some sequential interleaving of these transaction instances. rency and availability in order to recover the pleasant safety While serializability is highly desirable, it requires using properties afforded by serializability? Surprisingly, we are costly centralized locks [25] or complex version manage- able to answer this question in the negative. To see why, ment systems [10], which severely reduce the system’s avail- observe that a database program consists of two main com- able concurrency, especially in distributed environments ponents - a set of computations that includes transactions, where database state may be replicated or partitioned to SQL operations (e.g., selects and updates), locks, isolation- improve availability. In such environments, enforcing seri- level annotations, etc.; and a memory abstraction expressed alizability typically either requires coordination among all as a relational schema that defines the layout of tables and replicas whenever shared data is accessed or updated, or the relationship between them. The traditional candidates ensuring replicas always witness the same consistent order picked for repairing a serializability anomaly are the trans- of operations [16]. As a result, in most modern database sys- actions from the computational component: by injecting tems, transactions can be executed under weaker isolation additional concurrency control through the use of locks or levels, e.g. permitting them to observe updates of other com- isolation-strengthening annotations, developers can control mitted transactions during their execution [34, 38, 43, 48]. the degree of concurrency permitted, albeit at the expense Unfortunately, these weaker guarantees can result in serial- of performance and availability. izability anomalies, or behaviors that would not occur in a This paper investigates the under-explored alternative of serial execution. To illustrate, Figure 2 presents three con- transforming the program’s schema to reduce the number current executions of this program’s transaction instances of potentially conflicting accesses to shared state. For ex- that exhibit non-serializable behaviors. ample, by aggregating information found in multiple tables The execution on the left shows instances of the getSt into a single row on a single table, we can exploit built-in and setSet transactions. Following the order in which op- row-level atomicity properties to eliminate concurrency bugs erations execute (denoted by red arrows), observe that (S2) that arise because of multiple non-atomic accesses to differ- witnesses the update to a student’s email address, but (S1) ent table state. Row-level atomicity, a feature supported in does not see their updated name. This anomaly is known as most database systems, guarantees that other concurrently a non-repeatable read. The execution in the center depicts the executing transactions never observe partial updates to a 34
PLDI ’21, June 20ś25, 2021, Virtual, Canada Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannathan STUDENT st_id st_name st_em_id st_em_addr st_co_id st_co_avail st_reg Using the function uuid() ensures that a new record is in- COURSE_CO_ST_CNT_LOG co_id log_id co_st_cnt_log serted every time the transaction is called. These updates remove potential serializability anomalies by replacing the 1 getSt ( id ) : disjoint updates to fields in different tables from the original 2 x := select * from STUDENT where st_id = id // RS1 , RS2 , RS3 with a simple atomic row insertion. Notably, the refactored 1 setSt ( id , name , email ) : program can be shown to be a meaningful refinement of 2 update STUDENT set st_name = name , st_em_addr = email the original program, despite eliminating problematic seri- 3 where st_id = id // RU1 , RU2 alizability errors found in it. Program refinement ensures 1 regSt ( id , course ) : that the refactored program maintains all information main- 2 update STUDENT set st_co_id = course , st_co_avail = true , tained by the original program without exhibiting any new 3 st_reg = true where st_id = id // RU3 4 insert into COURSE_CO_ST_CNT_LOG values behaviour. 5 ( co_id = course , log_id = uuid () , co_st_cnt_log =1) // RU4 The program shown in Figure 3 is the result of several data- base schema refactorings [3, 21, 24], incremental changes to a database program’s data model along with correspond- Figure 3. Refactored transactions and database schemas ing semantic-preserving modifications to its logic. Manually searching for such a refactored program is unlikely to be particular row. Alternatively, it is possible to decompose data- successful. On one hand, the set of potential solutions is base state to minimize the number of distinct updates to a large [3], rendering any manual exploration infeasible. On field, for example by logging state changes via table inserts, the other hand, the process of rewriting an application for a rather than recording such changes via updates. The former (even incrementally) refactored schema is extremely tedious effectively acts as a functional update to a table. To be sure, and error-prone [55]. these transformations affect read and write performance to We have implemented a tool named Atropos that, given a database tables and change the memory footprint, but they database program, explores the space of its possible schema notably impose no additional synchronization costs. In scal- and program refactorings, and returns a new version with able settings such as replicated distributed environments, possibly many fewer concurrency bugs. The refactored pro- this is a highly favorable trade-off since the cost of global gram described above, for example, is automatically gener- concurrency control or coordination is often problematic ated by Atropos from the original shown in Figure 1. Figure 4 in these settings, an observation that is borne our in our presents the Atropos pipeline. A static analysis engine is experimental results. used to identify potential serializability anomalies in a given To illustrate the intuition behind our approach, consider program. The program is then pre-processed to extract the the database program depicted in Figure 3. This program components which are involved in at least one anomaly, in behaves like our previous example, despite featuring very order to put it into a form amenable for our analysis. Next, a different database schemas and transactions. The first of the refactoring engine applies a variety of transformations in an two tables maintained by the program, STUDENT, removes the attempt to eliminate the bugs identified by our static analysis. references to other tables from the original STUDENT table, in- Finally, the program is analyzed to eliminate dead code, and stead maintaining independent fields for the student’s email the refactored version is then reintegrated into the program address and their course availability. These changes make the from which it was extracted. original course and email tables obsolete, so they have been Transactional Program Refactored Program removed. In addition, the number of students in each course Static is now stored in a dedicated table COURSE_CO_ST_CNT_LOG. Anomaly Pre-proce Refactoring Post-proc ssor Engine essor Detector Each time the enrollment of a course changes, a new record is inserted into this table to record the change. Subsequent Schema Declaration Refactored Schema queries can retrieve all corresponding records in the table and aggregate them in the program itself to determine the Figure 4. Schematic overview of Atropos number of students in a course. The transactions in the refactored program are also modi- fied to reflect the changes in the data model. The transaction getSt now simply selects a single record from the student 3 Database Programs table to retrieve all the requested information for a student. We adopt a commonly-used model for database applica- The transaction setSt similarly updates a single record. Note tions [41, 42, 44], in which programs consist of a statically that both these operations are executed atomically, thus elim- known set of transactions that are comprised of a combina- inating the problematic data accesses in the original program. tion of control flow and database operations. The syntax of Similarly, regSt updates the student’s st_co_id field and in- our database programs is given in Figure 5. A program is serts a new record into the schema COURSE_CO_ST_CNT_LOG. defined in terms of a set of database schemas ( ), and a set 35
Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI ’21, June 20ś25, 2021, Virtual, Canada a database command is executed; it is used to resolve con- ∈ FldName ⊕ ∈ {+, −, ×, /} flicts among concurrent operations performed on the same ∈ SchmName ⊙ ∈ {, ≥ } ∈ TxnName ◦ ∈ {∧, ∨} elements, which can be used to define a linearization or ar- ∈ Arg ≔ ( ) { ; return } bitration order on updates [15]. Given a database state (Σ), ∈ Var ≔ : and a primary key ∈ id , it is possible to reconstruct each ∈ Val ≔ ⟨ : ⟩ field of a record , which we denote as Σ( . ). agg ∈ {sum, min, max} ( , ) ≔ Retrieving a record from a table generates a set of read ≔ | | ⊕ | ⊙ | ◦ | iter | agg( . ) | at ( . ) events, rd( , , ), which witness that the field of the record ≔ this. ⊙ | ◦ with the primary key ∈ id was accessed when the value ≔ := SELECT FROM WHERE | UPDATE SET = WHERE of the execution counter was . Similarly, a write event, ≔ | iterate(e) {c} | if(e) { } | skip | ; wr( , , , ), records that the field of record was assigned the value at timestamp . The timestamp (resp. record) Figure 5. Syntax of database programs associated with an event is denoted by (resp. ). Our semantics enforces record-level atomicity guarantees: transactions never witness intermediate (non-committed) updates to a record in a table by another concurrently ex- of transactions ( ). A database schema consists of a schema ecuting one. Thus, all updates to fields in a record from a name ( ) and a set of field names ( ). A database record ( ) database command happen atomically. This form of atomic- for schema is comprised of a set of value bindings to ’s ity is offered by most commercial database systems, and is fields. A database table is a set of records. Associated with easily realized through the judicious use of locks. Enforcing each schema is a non-empty subset of its fields that act as stronger multi-record atomicity guarantees is more challeng- a primary key. Each assignment to these fields identifies a ing, especially in distributed environments with replicated unique record in the table. In the following, we write id database state [6, 9, 20, 35, 57]. In this paper, we consider to denote the set of all possible primary key values for the behaviors induced when the database guarantees only a very schema . In our model, a table includes a record correspond- weak form of consistency and isolation that allows trans- ing to every primary key. Every schema includes a special actions to see an arbitrary subset of committed updates by Boolean field, ∈ FldName, whose value determines if other transactions. Thus, a transaction which accesses multi- a record is actually present in the table. This field allows us ple records in a table is not obligated to witness all updates to model DELETE and INSERT commands without explicitly performed by another transaction on these records. including them in our program syntax. To capture these behaviors, we use a visibility relation Transactions are uniquely named, and are defined by a between events, vis, that relates two events when one wit- sequence of parameters, a body, and a return expression. The nesses the other in its local view of the database at the time of body of a transaction ( ) is a sequence of database commands its creation. A local view is captured by the relation ⊆ Σ×Σ ( ) and control commands. A database command either modi- between database states, which is constrained as follows: fies or retrieves a subset of records in a database table. The (ConstructView) records retrieved by a database query are stored locally and str′ ⊆ str ∀ ′ ∈str′ ∀ ∈str ( = ′ ∧ = ′ ) ⇒ ( ∈ str′ ) can be used in subsequent commands. Control commands vis′ = vis |str′ cnt′ = cnt consist of conditional guards, loops, and return statements. (str′, vis′, cnt′ ) (str, vis, cnt) Both database commands (SELECT and UPDATE ) require an The above definition ensures that an event can only be explicit where clause ( ) to filter the records they retrieve or present in a local view, str′, if all other events on the same update. fld denotes the set of fields appearing in a clause . record with the same counter value are also present in Expressions ( ) include constants, transaction arguments, str′ (ensuring record-level atomicity). Additionally, the vis- arithmetic and Boolean operations and comparisons, iter- ibility relation permitted on the local view, vis′, must be ation counters and field accessors. The values of field consistent with the global visibility relation, vis. of records stored in a variable can be aggregated using Figure 6 presents the operational semantics of our lan- agg( . ), or accessed individually, using at ( . ). guage, which is defined by a small-step reduction relation, ⇒ ⊆ Σ × Γ × Σ × Γ, between tuples of data-store states 3.1 Data Store Semantics (Σ) and a set of currently executing transaction instances Database states Σ are modeled as a triple (str, vis, cnt), (Γ ⊆ × × (Var ⇀ id × )). A transaction instance is where str is a set of database events ( ) that captures the a tuple consisting of the unexecuted portion of the trans- history of all reads and writes performed by a program oper- action body (i.e., its continuation), the transaction’s return ating over the database, and vis is a partial order on those expression, and a local store holding the results of previously events. The execution counter, cnt, is an integer that rep- processed query commands. The rules are parameterized resents a global timestamp that is incremented every time over a program containing a set of transactions, txn . At 36
PLDI ’21, June 20ś25, 2021, Virtual, Canada Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannathan (txn-invoke) (txn-step) (txn-ret) ∈ Val ( ) { ; return } ∈ txn − Σ′, Δ′, ′ Σ, Δ, → ∉ Val Δ, ⇓ Σ, Γ ⇒ Σ, Γ∪ { : [ / ]; skip ; [ / ]; ∅} Σ, { : ; ; Δ}∪Γ ⇒ Σ′, { : ′ ; ; Δ′ }∪Γ Σ, { : skip; ; Δ}∪Γ ⇒ Σ, { : skip; ; Δ}∪Γ (seq) (skip) (cond-t) (cond-f) (iter) − Σ′, Δ′, ′′ Σ, Δ, → Δ, ⇓ true Δ, ⇓ false Δ, ⇓ Σ, Δ, ; ′ → − Σ′, Δ′, ′′ ; ′ Σ, Δ, skip; → − Σ, Δ, Σ, Δ, if( ) { } → − Σ, Δ, Σ, Δ, if( ) { } → − Σ, Δ, skip Σ, Δ, iterate( ) { } → − Σ, Δ, concat( , ) (select) Σ′ Σ 1 = {rd(cnt, , ) | ∈ id ∧ ∈ fld } (update) results = { ( , ⟨ : ⟩) | ∈ id ∧ Σ′ ( ) = ⟨ ′ : ′ ⟩ ∧ Σ′ Σ Δ, ( ⟨ ′ : ′ ⟩) ⇓ true ∧ : ⊆ ′ : ′ } = {wr(cnt, , , ) | ∈ id ∧ Σ′ ( ) = ⟨ : ⟩ ∧ 2 = {rd(cnt, , ′ ) | ( , ⟨ ′ : ′ ⟩) ∈ results ∧ ′ ∈ } Δ, ( ⟨ : ⟩) ⇓ true ∧ ( = ) ∈ = ∧ Δ, ⇓ } str′ = Σ.str ∪ 1 ∪ 2 vis′ = Σ.vis ∪ { ( , ′ ) | ′ ∈ 1 ∪ 2 ∧ ∈ Σ′ .str) } str′ = Σ.str ∪ vis′ = Σ.vis ∪ { ( , ′ ) | ′ ∈ ∧ ∈ Σ′ .str} − (str′, vis′, cnt + 1), Δ [ ↦→ results], skip Σ, Δ, := SELECT FROM WHERE → − (str′, vis′, cnt + 1), Δ, skip Σ, Δ, UPDATE SET = WHERE → Figure 6. Operational semantics of weakly-isolated database programs. every step, a new transaction instance can be added to the view. All updates are performed atomically, as the set of cor- set of currently running transactions via (txn-invoke). Al- responding write events all have the same timestamp value, ternatively, a currently running transaction instance can be however, other transactions are not obligated to see all the processed via (txn-step). Finally, if the body of a transac- effects of an update since their local view may only capture tion has been completely processed, its return expression a subset of these events. is evaluated via (txn-ret); the resulting instance simply records the binding between the transaction instance ( ) and 3.2 Anomalous Data Access Pairs its return value ( ). The semantics of commands are defined using a local We reason about concurrency bugs on transactions induced reduction relation (→) on database states, local states, and by our data store programming model using execution histo- commands. The semantics for control commands are straight- ries; finite traces of the form: Σ1, Γ1 ⇒ Σ2, Γ2 ⇒ · · · ⇒ Σ , Γ forward outside of the (iter) rule, which uses an auxiliary that capture interleaved execution of concurrently executing function concat( , ) to sequence copies of the command transactions. A complete history is one in which all trans- . Expression evaluation is defined using the big-step rela- actions have finished, i.e., the final Γ in the trace is of the form: { 1 : skip; 1, Δ1 } ∪ . . . ∪ { : skip; , Δ }. As a tion ⇓ ⊆ (Var ⇀ id × ) × × Val which, given a store shorthand, we refer to the final state in a history ℎ as ℎ fin . A holding the results of previous query commands, determines serial execution history satisfies two important properties: the final value of the expression. The full definition of ⇓ can be found in the supplementary material. Strong Atomicity: (∀ , ′ . cnt < cnt ′ ⇒ vis( , ′)) ∧ The semantics of database commands, given by the (select) ∀ , , . st( , ) ∧ (vis( , ) ⇒ vis( ′, ′′)) ′ ′′ ′ ′′ and (update) rules, expose the interplay between global and Strong Isolation: ∀ , ′, ′′ . st( , ′) ∧ vis( ′′, ′) ⇒ local views of the database. Both rules construct a local view vis( ′′, ). of the database (Σ ′ Σ) that is used to select or update the contents of records. Neither rule imposes any restric- The strong atomicity property prevents non-atomic inter- tions on Σ ′ other than the consistency constraints defined leavings of concurrently executing transactions. The first by (ConstructView). The key component of each rule is constraint linearizes events, relating timestamp ordering of how it defines the set of new events ( ) that are added to the events to visibility. The second generalizes this notion to database. In the select rule, 1 captures the retrievals that oc- multiple events, obligating all effects from the same trans- cur on database-wide scans to identify records satisfying the action (identified by the st relation) to be visible to another SELECT command’s where clause. In an abuse of notation, we if any of them are; in particular, any recorded event of a write Δ, (⟨ : ⟩) ⇓ as shorthand for Δ, [this.f/ ] ⇓ . transaction 1 that precedes an event in 2 requires all of 1′ 2 constructs the appropriate read events of these retrieved events to precede all of 2 ’s. records. The (update) rule similarly defines , the set of The strong isolation property prevents a transaction from write events on the appropriate fields of the records that observing the commits of other transactions once it begins satisfy the where clause of the UPDATE command under an execution. It does so through visibility constraints on a trans- arbitrary (but consistent) local view (Σ ′) of the global store action that require any event ′′ generated by any other (Σ). Both rules increment the local timestamp, and establish transaction that is visible to an event ′ generated by to new global visibility constraints reflecting the dependencies be visible to any event that precedes it in ’s execution. introduced by the database command, i.e., all the generated A serializability anomaly is an execution history with a read and write events depending upon the events in the local final state that violates at least one of the above constraints. 37
Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI ’21, June 20ś25, 2021, Virtual, Canada These sorts of anomalies capture when the events of a trans- COURSE 0 COURSE_ST_CNT_LOG 0 action instance are either not made visible to other events co_id co_avail co_st_cnt co_id co_log_id co_cnt_log 1 true 2 1 11 1 in totality (in the case of a violation of strong atomicity) 2 true 1 2 22 1 or which themselves witness different events (in the case 1 33 1 of a violation of strong isolation). Both kinds of anomalies STUDENT0 can be eliminated by identifying commands which gener- st_id st_name st_em_id st_em_addr st_co_id st_co_avail st_reg 100 Bob 1 Bob@host.com 1 true true ate sets of problematic events and altering them to ensure 200 Alice 2 Alice@host.com 1 true true atomic execution. Two events are executed atomically if they 300 Chris 3 Chris@host.com 2 true true witness the same set of events and they are both made visible to other events simultaneously, i.e. atomic( , ′) ≡ Figure 7. An example illustrating value correspondences. ∀ ′′ . (vis( , ′′) ⇒ vis( ′, ′′))∧(vis( ′′, ) ⇒ vis( ′′, ′)). Given a program , we define a database access pair ( ) 4.1 Database Containment as a quadruple ( 1, 1, 2, 2 ) where 1 and 2 are database commands from a transaction in , and 1 (resp. 2 ) is a Consider the tables in Figure 7, which are instances of the subset of the fields that are accessed by 1 (resp. 2 ). An schemas from Section 2. Note that every field of COURSE0 can access pair is anomalous if there is at least one execution in be computed from the values of some other field in either the execution history of P that results in an event generated the STUDENT0 or COURSE_ST_CNT_LOG0 tables: co_avail cor- responds to the value of the st_co_avail field of a record by 1 accessing a field 1 ∈ 1 which induces a serializability in STUDENT0 , while co_st_cnt can be recovered by sum- anomaly with another event generated by 2 accessing field ming up the values of the co_cnt_log field of the records in 2 ∈ 2 . An example of an anomalous access pair for the COURSE_ST_CNT_LOG0 whose co_id field has the same value program from in Section 2, is ( 1, {st_name}, 2, {em_addr}) as the original table. and ( 1, {st_name}, 2, {em_addr}); this pair contributes to The containment relation between a table (e.g. COURSE0 ) that program’s non-repeatable read anomaly from Figure 2. and a set of tables (e.g. STUDENT0 and COURSE_ST_CNT_LOG0 ) We now turn to the development of an automated static re- is defined using a set of mappings called value correspon- pair strategy that given a program and a set of anomalous dences [55]. A value correspondence captures how to com- access pairs produces a semantically equivalent program ′ pute a field in the contained table from the fields of the with fewer anomalous access pairs. In particular, we repair containing set of tables. Formally, a value correspondence programs by refactoring their database schemas in order to between field of schema and field ′ of schema ′ is benefit from record-level atomicity guarantees offered by defined as a tuple ( , ′, , ′, , ) in which: (i) a total record correspondence function, denoted by : id → id ′ , relates most databases, without introducing new observable behav- iors. We elide the details of how anomalous access pairs every record of any instance of to a set of records in any are discovered, but note that existing tools [13, 46] can be instance of ′ and (ii) a fold function on values, denoted by adapted for this purpose. Section 6 provides more details : Val → Val is used to aggregate a set of values. We say about how this works in Atropos. that a table is contained by a set of tables under a set of value correspondences , if accurately explains how to compute from , i.e. ⊑ ≡ ∀ ∈ fld . ∃( , ′, , ′, , ) ∈ . ∃ ′ ∈ . 4 Refactoring Database Programs ∀ ∈ id . ( . ) = ({ | ′ ∈ ( ) ∧ ′ ( ′ . ′ ) = }) In this section, we establish the soundness properties on For example, the table COURSE0 is contained in the set the space of database program refactorings and then intro- of tables {STUDENT0, COURSE_ST_CNT_LOG0 } under the pair duce our particular choice of sound refactoring rules. Similar of value correspondences, (COURSE, STUDENT, co_avail, refactorings are typically applied by developers when mi- st_co_avail, 1, any) and (COURSE, COURSE_ST_CNT_LOG, grating traditional database programs to distributed database co_st_cnt, co_cnt_log, 2, sum), where 1 (1) = {100, 200}, systems [28, 58]. Our approach to repair can be thought of 1 (2) = {300}, 2 (1) = {(1, 11), (1, 33)} and 2 (2) = {(2, 22)}. as automating this manual process in a way that eliminates The aggregator function any : Val → Val returns a non- serializability anomalies. deterministically chosen value from a set of values. The The correctness of our approach relies on being able to containment relation on tables is straightforwardly lifted to show that each program transformation maintains the invari- data store states, denoted by Σ ⊑ Σ ′, if all tables in Σ are ant that at every step in any history of a refactored program, contained by the set of tables in Σ ′. it is possible to completely recover the state of the data-store We define the soundness of our program refactorings us- for a corresponding history of the original program. To estab- ing a pair of refinement relations between execution histo- lish this property, we begin by formalizing the notion of a ries and between programs. An execution history ℎ ′ (where containment relation between tables. ′ ℎ fin = (Σ ′, Γ ′)) is a refinement of an execution ℎ (where 38
PLDI ’21, June 20ś25, 2021, Virtual, Canada Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannathan (intro ) (intro . ) transactions getSt and setSt to utilize a value correspon- ∉ RelNames = : ∉ ′ = : ∪ { } dence from em_addr to st_em_addr, moving email addresses , ( , ) ↩− → , ( ∪ { : ∅ }, ) → , ( { ′ } ∪ , ) , ( { } ∪ , ) ↩− to the STUDENT table, as described in Section 2. The select (intro ) commands S1 and S3 in getS remain unchanged after the ∉ ′ = { ( ) { [[ ]] ; return [[ ]] } | ( ) { ; return } ∈ } refactoring, as they do not access the affected table. However, → ∪ { }, ( , ′ ) , ( , ) ↩− the query S2, which originally accessed the EMAIL table is redirected to the STUDENT table. Figure 8. Refactoring Rules More generally, in order to take advantage of a newly ℎ fin = (Σ, Γ)), denoted by ℎ ′ ⪯ ℎ, if and only if Γ ′ and Γ added value correspondence , [[.]] must alter every query have the same collection of finalized transaction instances on the source table and field in to use the target table of and there is a set of value correspondences under which Σ instead, so that the new query accesses the same data as the is contained in Σ ′, i.e. Σ ⊑ Σ ′. Intuitively, any refinement of original. This rewrite has the general form: a history ℎ maintains the same set of records and spawns the same set of transaction instances as ℎ, with each instance [[ := SELECT FROM WHERE ]] ≡ producing the same result as it does in ℎ. Lastly, we define (1) := SELECT ′ FROM ′ WHERE redirect( , ) a refactored program ′ to be a refinement of the original program , denoted by ′ ⪯ , if the following conditions Intuitively, in order for this transformation to ensure R1, are satisfied: the redirect function must return a new where clause on the target table which selects a set of records corresponding (I) Every history ℎ ′ of ′ has a corresponding history ℎ to set selected by the original clause. in such that ℎ ′ is a refinement of ℎ. In order to preserve R2, program expressions also need to (II) Every serializable history ℎ of has a corresponding be rewritten to evaluate to the same value as in the original history ℎ ′ in ′ such that ℎ ′ is a refinement of ℎ. program. For example, observe that the return expression The first condition ensures that ′ does not introduce any in getSt is updated to reflect that the records held in the new behaviors over , while the second ensures that ′ does variable y now adhere to a different schema. not remove any desirable behavior exhibited by . The transformation performed in Figure 9 also rewrites the update (U2) of transaction setSt. In this case, the update 4.2 Refactoring Rules is rewritten using the same redirection strategy as (S2), so We describe Atropos’s refactorings using a relation ↩− →⊆ that it correctly reflects the updates that would be performed × × × , between programs and sets of value cor- by the original program to the EMAIL record. respondences. The rules in Figure 8 are templates of the Taken together, 1 − 3 are sufficient to ensure that a three categories of transformations employed by Atropos. particular instance of intro is sound2 : These categories are: (1) adding a new schema to the pro- gram, captured by the rule (intro ); (2) adding a new field Theorem 4.1. Any instance of intro whose instantiation to an existing schema , captured by rule (intro . ); and, of [[·]] satisfies 1 − 3 is guaranteed to produce a refactored (3) relocating certain data from one table to another while program that refines the original, i.e. modifying the way it is accessed by the program, captured → ( ∪ { }, ′) ⇒ ′ ⪯ ∪{ } ∀ , ′, , . ( , ) ↩− by the rule (intro ). The refactorings represented by (intro ) introduce a new value correspondence , and modify the body and re- Although our focus has been on preserving the semantics of turn expressions of a programs transactions via a rewrite refactored programs, note that as a direct consequence of our function, [[.]] . A particular instantiation of [[.]] must en- definition of program refinement, this theorem implies that sure the same data is accessed and modified by the resulting sound transformations do not introduce any new anomalies. program, in order to guarantee that the refactored program We now present the instantiations of intro used by refines the original. At a high-level, it is sufficient for [[·]] Atropos, explaining along the way how they ensure R1-R3. to ensure the following relationship between the original ( ) and refactored programs ( ′) : 4.2.1 Redirect Rule. Our first refactoring rule is param- (R1) ′ accesses the same data as , which may be main- eterized over the choice of schemas and fields and uses tained by different schemas; the aggregator any. Given data store states Σ and Σ ′, the (R2) ′ returns the same final value as ; record correspondence is defined as: ⌈ ˆ⌉ ( ) ≡ { ′ | ′ ∈ ′ ∧∀ ′ ′ ˆ (R3) and, ′ properly updates all data maintained by . id ∈ id ∀ . Σ( . ) ⇓ ⇒ Σ ( . ( )) ⇓ }. In essence, To see how a rewrite function might ensure R1 to R3, con- 2A complete formalization of all three refactoring rules, their correctness sider the original (top) and refactored (bottom) programs criteria, and proofs of soundness is presented in the extended version of presented in Figure 9. This example depicts a refactoring of this paper [47]. 39
Repairing Serializability Bugs in Distributed Database Programs via Automated Schema Refactoring PLDI ’21, June 20ś25, 2021, Virtual, Canada 1 getSt ( id ) : 1 setSt ( id , name , email ) : 2 x := select * from STUDENT where st_id = id // S1 2 x := select st_em_id from STUDENT where st_id = id // S4 3 y := select em_addr from EMAIL where em_id = x . st_em_id // S2 3 update STUDENT set st_name = name where st_id = id // U1 4 z := select co_avail from COURSE 4 update EMAIL set em_addr = email 5 where co_id = x . st_co_id // S3 5 where em_id = x . st_em_id // U2 6 return ( y . em_addr ) 6 return 0 intro (EMAIL, STUDENT, em_addr, st_em_addr, ⌈ ˆ0 ⌉, any) intro (EMAIL, STUDENT, em_addr, st_em_addr, ⌈ ˆ0 ⌉, any) ↩−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−→ ↩−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−→ 1 getSt ( id ) : 1 setSt ( id , name , email ) : 2 x := select * from STUDENT where st_id = id // S1 2 x := select st_em_id from STUDENT where st_id = id // S4 3 y := select st_em_addr from STUDENT 3 update STUDENT set st_name = name where st_id = id // U1 4 where st_em_id = x . st_em_id // S2 ' 4 update STUDENT set st_em_addr = email 5 z := select co_avail from COURSE 5 where st_em_id = x . st_em_id // U2 ' 6 where co_id = x . st_co_id // S3 6 return 0 7 return ( y . st_em_addr ) Figure 9. A single program refactoring step, where ˆ0 (EMAIL.em_addr) = STUDENT.st_em_addr the lifted function ˆ identifies how the value of the primary 4.2.2 Logger Rule. Unfortunately, instantiating intro key of a record can be used to constrain the value of is not so straightforward when we want to utilize value cor- field ˆ( ) in the target schema to recover the set of records respondences with more complicated aggregation functions corresponding to , i.e. ( ). The record correspondences than any. To see why, consider how we would need to mod- from Section 4.1 were defined in this manner, where ify an UPDATE when ≡ sum is used. In this case, our rule ˆ1 (COURSE.co_id) ≡ STUDENT.st_co_id , and transforms the program to insert a new record corresponding to each update performed by the original program. Hence, ˆ2 (COURSE.co_id) ≡ COURSE_CO_ST_CNT_LOG.co_id. the set of corresponding records in the target table always Defining the record correspondence this way ensures that if a grows and cannot be statically identified. record is selected in Σ, the corresponding set of records in Σ ′ We enable these sorts of transformations by using log- can be determined by identifying the values that were used to ging schema for the target schema. A logging schema for select , without depending on any particular instance of the source schema and the field is defined as follows: (i) the tables. Our choice of record correspondence function makes target schema ( ) has a primary key field, correspond- the definition of [[·]] for select statements a straightforward ing to every primary key field of the original schema ( ); instantiation of (1) with the following definition of redirect: (ii) the schema has one additional primary key field, denoted by .log_id, which allows a set of records in to Û represent each record in ; and (iii) the schema has a redirect( , ⌈ ˆ⌉) ≡ this. ˆ( ) = [ ]exp (2) single field corresponding to the original field . , denoted ∈ fld by . ′. The one wrinkle in this definition of redirect is that it is Intuitively, a logging schema captures the history of up- only defined when the where clause is well-formed, i.e. dates performed on a record, instead of simply replacing old only consists of conjunctions of equality constraints on values with new ones. Program-level aggregators can then primary key fields. The expressions used in such a constraint be utilized to determine the final value of each record, by is denoted by [ ]exp . As an example, the where clause observing all corresponding entries in the logging schema. of command (S2) in Figure 9 (left) is well-formed, where The schema COURSE_CO_ST_CNT_LOG from Section 2 is an [em_id]exp ≡ x.st_e_id. However, the where clause in example of a logging schema for the source schema and field (S2′) after the refactoring step is not well-formed, since it COURSE.co_st_cnt. does not constrain the primary key of the STUDENT table. Under these restrictions, we can define an implementa- This restriction ensures that only queries that access a single tion of [[·]] for the logger rule using sum as an aggregator. record of the original table will be rewritten. Expressions This refactoring also uses a lifted function ⌈ ˆ⌉ for its value using variables containing the results of such queries are correspondence, which allows [[·]] to reuse our earlier defi- rewritten by substituting the source field name with the nition of redirect. We define [[·]] on accesses to to use target field name, e.g. [[at1 ( . )]] ≡ at1 ( . ′). program-level aggregators, e.g. [[at1 ( . )]] ≡ sum( . ′). Redirecting updates is similarly defined using the defini- Finally, the rewritten UPDATE commands simply need to tion of redirect( , ) from 2: log any updates to the field , so its original value can be recovered in the transformed program, e.g. [[UPDATE SET = WHERE ]] ≡ UPDATE ′ SET ( ′ = [[ ]] ) WHERE redirect( , ) [[UPDATE SET = + at1 ( . ) WHERE ]] ≡ UPDATE ′ SET ′ = [[ ]] WHERE redirect( , ) ∧ ′ .log_id = uuid(). 40
PLDI ’21, June 20ś25, 2021, Virtual, Canada Kia Rahmani, Kartik Nagar, Benjamin Delaware, and Suresh Jagannathan Having introduced the particular refactoring rules instan- 1 regSt ( id , course ) : tiated in Atropos, we are now ready to establish the sound- 2 update STUDENT set st_co_id = course , st_reg = true 3 where st_id = id // U3 ness of those refactorings: 4 x := select co_st_cnt from COURSE 5 where co_id = course // S5 Theorem 4.2. The rewrite rules described in this section sat- 6 update COURSE set co_st_cnt = x . co_st_cnt +1 7 where co_id = course // U4 .1 isfy the correctness properties (R1), (R2) and (R3). 8 update COURSE set co_avail = true 9 where co_id = course // U4 .2 Corollary 4.3. (Soundness) Any sequence of refactorings per- formed by Atropos is sound, i.e. the refactored program is a intro (COURSE, STUDENT, co_avail, st_co_avail, ⌈ ˆ1 ⌉, any) ↩−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−→ refinement of the original program. 1 regSt ( id , course ) : 2 update STUDENT set st_co_id = course , st_reg = true Proof. Direct consequence of theorems 4.1 and 4.2. □ 3 where st_id = id // U3 4 x := select co_st_cnt from COURSE where co_id = course // S5 5 Repair Procedure 5 6 update COURSE set co_st_cnt = x . co_st_cnt +1 Figure 10 presents our algorithm for eliminating serializ- 7 where co_id = course // U4 .1 8 update STUDENT set st_co_avail = true ability anomalies using the refactoring rules from the previ- 9 where st_co_id = course // U4 .2 ' ous section. The algorithm (repair) begins by applying an intro COURSE_CO_ST_CNT_LOG anomaly detector O to a program to identify a set of anoma- ↩−−−−−−−−−−−−−−−−−−−−−−→ . . . intro (COURSE, COURSE_ST_CNT_LOG, co_st_cnt, co_cnt_log, ⌈ ˆ2 ⌉,sum) lous access pairs. As an example, consider regSt from our ↩−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−→ running example. For this transaction, the anomaly oracle 1 regSt ( id , course ) : identifies two anomalous access pairs: 2 update STUDENT set st_co_id = course , st_reg = true 3 where st_id = id // U3 (U3, {st_co_id, st_reg}, U4, {co_avail}) ( 1 ) 4 x := select co_st_cnt from COURSE 5 where co_id = course // S5 (S5, {co_st_cnt}, U4, {co_st_cnt}) ( 2 ) 6 insert into COURSE_CO_ST_CNT_LOG values // U4 .1 ' 7 ( co_id = course , log_id = uuid () , co_st_cnt_log =1) The first of these is involved in the dirty read anomaly from 8 update STUDENT set st_co_avail = true 9 where st_co_id = course // U4 .2 ' Section 2, while the second is involved in the lost update anomaly. Figure 11. Repair steps of transaction regSt Function : repair( ) 1 ← O ( ); ← pre_process( , ) repair them one by one using try_repair. This function 2 for ∈ do attempts to eliminate a given anomaly in two different ways; 3 if try_repair( , ) = ′ then ← ′ either by merging anomalous database commands into a sin- 4 return post_process( ) gle command, and/or by removing one of them by making it obsolete. In the remainder of this section, we present these Function : try_repair( , ) 1 1 ← . 1 ; 2 ← . 2 two strategies in more detail, using the running example 2 if same_kind( 1 , 2 ) then from Figure 11. 3 if same_schema( 1 , 2 ) then We first explain the merging approach. Two database com- 4 return try_merging( , 1 , 2 ) mands can only be merged if they are of the same kind (e.g. 5 else if try_redirect( , 1 , 2 ) = ′ then both are selects and if they both access the same schema. 6 return try_merging( ′, 1 , 2 ) These conditions are checked in lines 2-3. Function try_merge 7 return try_logging( , 1 , 2 ) attempts to merge the commands if it can establish that their where clauses always select the exact same set of records, i.e. Figure 10. The repair algorithm condition (R1) described in Section 4.2. Unfortunately, database commands involved in anomalies The repair procedure next performs a pre-processing phase, are rarely on the same schema and cannot be merged as they where database commands are split into multiple commands originally are. Using the refactoring rules discussed earlier, such that each command is involved in at most one anoma- Atropos attempts to introduce value correspondences so lous access pair. For example, the first step of repairing the that the anomalous commands are redirected to the same regSt transaction is to split command U4 into two update table in the refactored program and thus mergeable. This is commands, as shown in Figure 11 (top). Note that we only captured by the call to the procedure try_redirect. This perform this step if the split fields are not accessed together in procedure first introduces a set of fields into the schema other parts of the program; this is to ensure that the splitting accessed by 1 , each corresponding a field accessed by 2 . does not introduce new unwanted serializability anomalies. Next, it attempts to introduce a sequence of value correspon- After pre-processing, the algorithm iterates over all de- dences between the two schemas using the redirect rule, tected anomalous access pairs ( ) and greedily attempts to such that 2 is redirected to the same table as 1 . The record 41
You can also read