Ferrite: A Judgmental Embedding of Session Types in Rust
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Ferrite: A Judgmental Embedding of Session Types in Rust RUOFEI CHEN, Independent Researcher, Germany arXiv:2009.13619v3 [cs.PL] 25 Mar 2021 STEPHANIE BALZER, Carnegie Mellon University, USA This paper introduces Ferrite, a shallow embedding of session types in Rust. In contrast to existing session type libraries and embeddings for mainstream languages, Ferrite not only supports linear session types but also shared session types. Shared session types allow sharing (aliasing) of channels while preserving session fidelity (preservation) using type modalities for acquiring and releasing sessions. Ferrite adopts a propo- sitions as types approach and encodes typing derivations as Rust functions, with the proof of successful type-checking manifesting as a Rust program. We provide an evaluation of Ferrite using Servo as a practical example, and demonstrate how safe communication can be achieved in the canvas component using Ferrite. CCS Concepts: • Theory of computation → Linear logic; Type theory; • Software and its engineering → Domain specific languages; Concurrent programming languages. Additional Key Words and Phrases: Session Types, Rust, DSL ACM Reference Format: Ruofei Chen and Stephanie Balzer. 2021. Ferrite: A Judgmental Embedding of Session Types in Rust. In Proceedings of International Conference on Functional Programming (ICFP 2021). ACM, New York, NY, USA, 36 pages. 1 INTRODUCTION Message-passing concurrency is a dominant concurrency paradigm, adopted by mainstream lan- guages such as Erlang, Scala, Go, and Rust, putting the slogan “to share memory by communicat- ing rather than communicating by sharing memory”[Gerrand 2010; Klabnik and Nichols 2018] into practice. In this setting, messages are exchanged over channels, which can be shared among sev- eral senders and recipients. Figure 1 provides a simplified example in Rust. It sketches the main communication paths in Servo’s canvas component [Mozilla 2021]. Servo is a browser engine un- der development that uses message-passing for heavy task parallelization. The canvas provides 2D graphic rendering services, allowing its clients to create new canvases and perform operations on a canvas such as moving the cursor and drawing shapes. The canvas component is implemented by the CanvasPaintThread, whose function start con- tains the main communication loop running in a separate thread (lines 10–20). This loop processes client requests received along canvas_msg_receiver and create_receiver, which are the receiv- ing endpoints of the channels created prior to spawning the loop (lines 8–9). The channels are typed with the enumerations ConstellationCanvasMsg and CanvasMsg, defining messages for creat- ing and terminating the canvas component and for executing operations on an individual canvas, respectively. When a client sends a message that expects a response from the recipient, such as GetTransform and IsPointInPath (lines 2–3), it sends a channel along with the message to be used by the recipient to send back the result. Canvases are identified by an id, which is generated upon canvas creation (line 19) and stored in the thread’s hash map named canvases (line 5). Should a client request an invalid id, for example after prior termination and removal of the canvas (line 16), the failed assertion expect("Bogus canvas id") (line 23) will result in a panic!, causing the canvas component to crash and subsequent calls to fail. ICFP 2021, 2021, Virtual 2021. ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. . . $15.00 1
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer 1 enum CanvasMsg { Canvas2d(Canvas2dMsg, CanvasId), Close(CanvasId), ... } 2 enum Canvas2dMsg { LineTo(Point2D), GetTransform(Sender), 3 IsPointInPath(f64, f64, FillRule, IpcSender), ... } 4 enum ConstellationCanvasMsg { Create { id_sender: Sender, size: Size2D } } 5 struct CanvasPaintThread { canvases: HashMap < CanvasId, CanvasData >, ... } 6 impl CanvasPaintThread { ... 7 fn start() -> ( Sender, Sender ) 8 { let ( msg_sender, msg_receiver ) = channel(); 9 let ( create_sender, create_receiver ) = channel(); 10 thread::spawn( move || { loop { select! { 11 recv ( canvas_msg_receiver ) -> { ... 12 CanvasMsg::Canvas2d ( message, canvas_id ) => { ... 13 Canvas2dMsg::LineTo( point ) => self.canvas(canvas_id).move_to(point), 14 Canvas2dMsg::GetTransform( sender ) => 15 sender.send( self.canvas(canvas_id).get_transform() ).unwrap(), ... } 16 CanvasMsg::Close ( canvas_id ) => canvas_paint_thread.canvases.remove(&canvas_id) } 17 recv ( create_receiver ) -> { ... 18 ConstellationCanvasMsg::Create { id_sender, size } => { 19 let canvas_id = ...; self.canvases.insert( canvas_id, CanvasData::new(size, ...) ); 20 id_sender.send(canvas_id); } } } } }); 21 ( create_sender, msg_sender ) } 22 fn canvas ( &mut self, canvas_id: CanvasId ) -> &mut CanvasData { 23 self.canvases.get_mut(&canvas_id).expect("Bogus canvas id") } } Fig. 1. Message-passing concurrency in Servo’s canvas component (simplified for illustration purposes). The code in Figure 1 uses a clever combination of enumerations to type channels and ownership to rule out races on the data sent along channels. Nonetheless, Rust’s type system is not expressive enough to enforce the intended protocol of message exchange and existence of a communication partner. The latter is a consequence of Rust’s type system being affine, which permits weakening, i.e., “dropping of a resource”. The dropping or premature closure of a channel, however, can result in a proliferation of panic! and thus cause an entire application to crash. This paper introduces Ferrite, a shallow embedding of session types in Rust. Session types [Honda 1993; Honda et al. 1998, 2008] were introduced to express the protocols of message exchange and to enforce their adherence at compile-time. Session types have a logical foundation by a Curry- Howard correspondence between linear logic and the session-typed -calculus [Caires and Pfenning 2010; Caires et al. 2016; Toninho 2015; Toninho et al. 2013; Wadler 2012], resulting in a linear treat- ment of channels and thus assurance of a communication partner. To address the limitations of an exclusively linear treatment of channels, shared session types were introduced [Balzer and Pfenning 2017; Balzer et al. 2018, 2019]. Shared session types support safe sharing (i.e., aliasing) of channels and accommodate multi-client scenarios, such as the one in Figure 1, that are ruled out by purely linear session types. For example, using linear and shared session types we can capture the implicit protocols in Figure 1 as follows: Canvas = ↑SL N{ LineTo : Point2D ⊲ ↓SL Canvas, GetTransform : Transform2D ⊳ ↓SL Canvas, IsPointInPath : (f64, f64, FillRule) ⊲ bool ⊳ Canvas, ...} ConstellationCanvas = ↑SL Size2D ⊲ Canvas ⊳ ↓SL ConstellationCanvas The shared session type Canvas prescribes the protocol for performing operations on an in- dividual canvas, and the shared session type ConstellationCanvas prescribes the protocol for creating a new canvas. We use the language SILLR , a formal session type language based on SILLS [Balzer and Pfenning 2017] that we extend with Rust-like constructs, to describe the formal specification of Ferrite. Table 1 provides an overview of SILLR ’s connectives. These are the usual linear connectives for receiving and sending values and channels ( ⊲ , ⊳ , ⊸, ⊗) as well as external and internal choice (N, ⊕). An external choice allows a client to choose among several options, whereas an internal choice leaves the decision to the provider. For example, Canvas provides the client the choice between LineTo to draw a line, GetTransform to get the current transformation, 2
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual Table 1. Overview and semantics of session types in SILLR and Ferrite. SILLR Ferrite Description for Provider End Terminate session. ⊲ ReceiveValue Receive value of type , then continue as session type . ⊳ SendValue Send value of type , then continue as session type . ⊸ ReceiveChannel Receive channel of session type , then continue as session type . ⊗ SendChannel Send channel of session type , then continue as session type . N{ : } ExternalChoice< HList! [ ] > Receive label inl or inr, then continue as session type or , resp. ⊕ { : } InternalChoice< HList! [ ] > Send label inl or inr, then continue as session type or , resp. ↑SL LinearToShared Accept an acquire, then continue as a linear session type . ↓SL SharedToLinear Initiate a release, then continue as a shared session type . and IsPointInPath to determine whether a 2D point is in a path. Readers familiar with classical linear logic session types [Wadler 2012] may notice the absence of linear negation. SILLR adopts an intuitionistic, sequent calculus-based formulation [Caires and Pfenning 2010], which avoids ex- plicit dualization of a connective by providing left and right rules. SILLR also comprises connectives to safely share channels (↑SL , ↓SL ). To ensure safe communi- cation, multiple clients must interact with the shared component in mutual exclusion of each other. To this end, an acquire-release semantics is adopted for shared components such that a shared component must first be acquired prior to any interaction. When a client acquires a shared com- ponent by sending an acquire request along the component’s shared channel, it obtains a linear channel to the component, becoming its unique owner. Once the client is done interacting with the component, it releases the linear channel, relinquishing the ownership and being left only with a shared channel to the component. Key to type safety is to establish acquire-release not as a mere programming idiom but to manifest it in the type structure [Balzer and Pfenning 2017], such that session types prescribe when to acquire and when to release. This is achieved with the modalities ↑SL and ↓SL , denoting the begin and end of a critical section, respectively. For example, the session type ConstellationCanvas prescribes that a client must first acquire the canvas component before it can ask for a canvas to be created, by sending values for the canvas’ size (Size2D). The compo- nent then sends back to the client a shared channel to the new canvas (Canvas), initiates a release (↓SL ), and recurs to be available to the next client. Ferrite is not the first session type embedding for a general purpose language, but other em- beddings exist for languages such as Java [Hu et al. 2010, 2008], Scala [Scalas and Yoshida 2016], Haskell [Imai et al. 2010; Lindley and Morris 2016; Pucella and Tov 2008; Sackman and Eisenbach 2008], OCaml [Imai et al. 2019; Padovani 2017], and Rust [Jespersen et al. 2015; Kokke 2019]. Fer- rite, however, is the first embedding that supports both linear and shared session types, with proto- col adherence guaranteed statically by the Rust compiler. Using Ferrite, the session types Canvas and ConstellationCanvas can be declared in Rust as follows: define_choice! { CanvasOps ; LineTo: ReceiveValue < Point2D, Z >, GetTransform: SendValue < Transform2D, Z >, IsPointInPath: ReceiveValue < ( f64, f64, FillRule ), SendValue < bool, Z > >, ... } type Canvas = LinearToShared < ExternalChoice < CanvasOps > >; type ConstellationCanvas = LinearToShared < ReceiveValue < Size2D, SendValue < SharedChannel < Canvas >, Z > > >; Table 1 provides a mapping between SILLR and Ferrite constructs. As we discuss in detail in Sections 2 and 4, Ferrite introduces LinearToShared as a recursive shared session type, and uses the type Z to denote the recursion point combined with a release. 3
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer Another distinguishing characteristics of Ferrite is its propositions as types approach. Building on the Curry-Howard correspondence between linear logic and the session-typed -calculus [Caires and Pfenning 2010; Wadler 2012], Ferrite encodes SILLR typing judgments and derivations as Rust functions. A successfully type-checked Ferrite program yields a Rust program that is the actual SILLR typing derivation and thus the proof of protocol adherence. Ferrite moreover makes several technical contributions to emulate in Rust advanced features from functional languages, such as higher-kinded types, by a skillful combination of traits (type classes) and associated types (type families). For example, Ferrite supports recursive session types in this way, which are limited to recursive structs of a fixed size in plain Rust. A combination of type-level natural numbers with ideas from profunctor optics [Pickering et al. 2017] are also used to support named channels and labeled choices for the DSL. We adopt the idea of lenses [Foster et al. 2007] for selecting and updating individual channels in an arbitrary-length linear context. Simi- larly, we use prisms [Pickering et al. 2017] for selecting a branch out of arbitrary-length choices. Whereas Padovani [2017] has previously explored the use of n-ary choice through extensible vari- ants available only in OCaml, we are the first to connect n-ary choice to prisms and non-native implementation of extensible variants. Remarkably, the Ferrite code base remains entirely within the safe fragment of Rust, with no direct use of unsafe Rust features. Given its support of both linear and shared session types, Ferrite is capable of expressing any ses- sion type program in Rust. We substantiate this claim by providing an implementation of Servo’s production canvas component with the communication layer done entirely within Ferrite. We re- port on our findings, including benchmark results in Section 6. In summary, Ferrite is an embedded domain-specific language (EDSL) for writing session type programs in Rust, with the following key contributions: • Judgmental embedding of custom typing rules in a host language with resulting program carry- ing the proof of successful type checking • Shared session types using adjoint modalities for acquiring and releasing sessions • Arbitrary-length choice in terms of prisms and extensible variants • Empirical evaluation based on a full re-implementation of Servo’s canvas component in Ferrite Outline: Section 2 provides a summary of the key ideas underlying Ferrite, with subsequent sections refining those. Section 3 introduces technical background on the Ferrite type system, fo- cusing on its judgmental embedding and enforcement of linearity. Section 4 explains how Ferrite addresses Rust’s limited support of recursive data types to allow the definition of arbitrary re- cursive and shared session types. Section 5 describes the implementation of n-ary choice using prisms and extensible variants. Section 6 provides an evaluation of the Ferrite library with a re- implementation of the Servo canvas component in Ferrite, and Section 7 reviews related and future work. Ferrite is released as the ferrite-session Rust crate. An anonymized version of Ferrite’s source code with examples is provided as supplementary material. All typing rules and their encoding are included in the appendix. We plan to submit Ferrite as an artifact. 2 KEY IDEAS This section highlights the key ideas underlying Ferrite. Subsequent sections provide further de- tails. 2.1 Judgmental Embedding In SILLR , the formal session type language that we use in this paper to study linear and shared session types, a program type-checks if there exists a derivation using the SILLR typing rules. Central to a typing rule is the notion of a typing judgment. For SILLR , we use the judgment 4
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual Table 2. Judgmental embedding of SILLR in Ferrite. SILLR Ferrite Description Γ; · ⊢ Session Typing judgment for top-level session (i.e., closed program). Γ; Δ ⊢ PartialSession Typing judgment for partial session. Δ C: Context Linear context; explicitly encoded. Γ - Structural / Shared context; delegated to Rust, but with clone semantics. A: Protocol Session type. Γ; Δ ⊢ expr :: to denote that the expression expr has session type , given the typing of the structural and linear channel variables free in contexts Γ and Δ, respectively. Γ is a structural context, which permits exchange, weakening, and contraction. Δ is a linear context, which only permits exchange but neither weakening nor contraction. The significance of the absence of weakening and contraction is that it becomes possible to “drop a resource” (premature closure) and “duplicate a resource” (aliasing), respectively. For example, to type-check session termination, SILLR defines the following typing rules: Γ ; Δ ⊢ K :: (T- L ) (T- R ) Γ ; Δ, : ⊢ wait ; K :: Γ ; · ⊢ terminate :: As mentioned in the previous section, we get a left and right rule for each connective because SILLR adopts an intuitionistic, sequent calculus-based formulation [Caires and Pfenning 2010]. Whereas the left rule describes the session from the point of view of the client, the right rule describes the session from the point of view of the provider. We read the rules bottom-up, with the meaning that the premise denotes the continuation of the session. The left rule (T- L ) indicates that the client is waiting for the linear session offered along channel to terminate, and continues with its continuation K once is closed. The linear channel is no longer available to the continuation due to the absence of contraction. The right rule (T- R ) indicates termination of the provider and thus has no continuation. The absence of weakening forces the linear context Δ to be empty, making sure that all resources are consumed. Given the notions of typing judgment, typing rule, and typing derivation, we can get the Rust compiler to type-check SILLR programs by encoding these notions as Rust programs. The basic idea underlying this encoding can be schematically described as follows: Γ ; Δ2 ⊢ cont :: 2 fn expr < ... > ( cont : PartialSession < C2, A2 > ) Γ ; Δ1 ⊢ expr; cont :: 1 -> PartialSession < C1, A1 > On the left we show a SILLR typing rule, on the right its encoding in Ferrite. Ferrite encodes a SILLR typing judgment as the Rust type PartialSession, with C representing the linear context Δ and A representing the session type . A SILLR typing rule for an expression expr, is encoded as a Rust function that accepts a PartialSession and returns a PartialSession. The encoding uses a continuation passing-style representation, with the return type being the conclu- sion of the rule and the argument type being its premise. This representation naturally arises from the sequent calculus-based formulation of SILLR . A Rust program calling expr is akin to obtain- ing a certificate of protocol correctness. The returned PartialSession are proof-carrying Ferrite programs, with static guarantee of protocol adherence. Table 2 provides an overview of the mapping between SILLR notions and their Ferrite encoding; Section 3.1 elaborates on them further. Whereas Ferrite explicitly encodes the linear context Δ, it delegates the handling of the structural and shared context Γ to Rust, with the obligation that shared channels implement Rust’s Clone trait to permit contraction. To type a closed program, 5
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer Ferrite moreover defines the type Session, which stands for a SILLR judgment with an empty linear context. 2.2 Recursive and Shared Session Types Rust’s support for recursive types is limited to recursive struct definitions of a known size. To circumvent this restriction and support arbitrary recursive session types, Ferrite introduces a type-level fixed-point combinator Rec to obtain the fixed point of a type function F. Since Rust lacks higher-kinded types such as Type → Type, we use defunctionalization [Reynolds 1972; Yallop and White 2014] by accepting any Rust type F implementing the trait RecApp with a given associated type F::Applied, as shown below. Section 4.1 provides further details. trait RecApp < X > { type Applied; } struct Rec < F: RecApp < Rec < F > > > { unfold: Box < F::Applied > } Recursive types are also vital for encoding shared session types. In line with [Balzer et al. 2019], Ferrite restricts shared session types to be recursive, making sure that a shared component is continuously available. To guarantee preservation, recursive session types must be strictly equi- synchronizing [Balzer and Pfenning 2017; Balzer et al. 2019], requiring an acquired session to be released to the same type at which it was previously acquired. Ferrite enforces this invariant by defining a specialized trait SharedRecApp which omits an implementation for End. trait SharedRecApp < X > { type Applied; } trait SharedProtocol { ... } struct SharedToLinear < F > { ... } struct LinearToShared < F: SharedRecApp < SharedToLinear > > { ... } struct SharedChannel < S: SharedProtocol > { ... } Ferrite achieves safe communication for shared sessions by imposing an acquire-release disci- pline [Balzer and Pfenning 2017] on shared sessions, establishing a critical section for the linear portion of the process enclosed within the acquire and release. SharedChannel denotes the shared process running in the background, and clients with a reference to it can acquire an exclusive linear channel to communicate with it. As long as the the linear channel exists, the shared pro- cess is locked and cannot be acquired by any other client. With the strictly equi-synchronizing constraint in place, the now linear process must eventually be released (SharedToLinear) back to the same shared session type at which it was previously acquired, giving turn to another client waiting to acquire. Section 4.2 provides further details on the encoding. 2.3 N-ary Choice and Linear Context Ferrite implements n-ary choices and linear typing contexts as extensible sums and products of session types, respectively. Ferrite uses heterogeneous lists [Kiselyov et al. 2004] to annotate a list of session types of arbitrary length. The notation HList![ 0, 1, ..., −1 ] denotes a heterogeneous list of N session types, with being the session type at the -th position of the list. The HList! macro acts as syntactic sugar for the heterogeneous list, which in its raw form is encoded as ( 0, ( 1 , (..., ( −1 )))). Ferrite uses the Rust tuple constructor (,) for HCons, and unit () for HNil. The heterogeneous list itself can be directly used to represent an n-ary product. Using an associated type, the list can moreover be transformed into an n-ary sum. One disadvantage of using heterogeneous lists is that its elements have to be addressed by po- sition rather than a programmer-chosen label. We recover labels for accessing list elements using optics [Pickering et al. 2017]. More precisely, Ferrite uses lenses [Foster et al. 2007] to access a channel in a linear context and prisms to select a branch of a choice. We further combine the op- tics abstraction with de Bruijn levels and implement lenses and prisms using type level natural numbers. Given an inductive trait definition of natural numbers as zero (Z) and successor (S), 6
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual a natural number N implements the lens to access the N-th element in the linear context, and the prism to access the N-th branch in a choice. Schematically the lens encoding can be captured as follows: fn expr < ... > Γ ; Δ, : ′ ⊢ K :: ′ ( l: N, cont: PartialSession < C1, A2 > ) -> PartialSession < C2, A1 > Γ ; Δ, : ⊢ expr ; K :: where N : ContextLens < C1, B1, B2, Target=C2 > We note that the index N amounts to the type of the variable l that the programmer chooses as a name for a channel in the linear context. Ferrite takes care of the mapping, thus supporting random access to programmer-named channels. Section 3.2 provides further details, including the support of higher-order channels. Similarly, the use of prism allows choice selection in constructs such as offer_case to be encoded as follows: fn offer_case < ... > Γ; Δ ⊢ K :: ( l: N, cont: PartialSession < C, A > ) -> PartialSession < C, InternalChoice > Γ; Δ ⊢ offer_case ; K :: ⊕{..., : , ...} where N : Prism < Row, Elem=A > Ferrite maps a choice label to a constant having the singleton value of a natural number N, which implements the prism to access the N-th branch of a choice. In addition to prism, Ferrite implements a version of extensible variants [Morris 2015] to support polymorphic operations on arbitrary sums of session types representing choices. Finally, the define_choice! macro is used as a helper macro to export type aliases as programmer-friendly identifiers. More details of the choice implementation are described in Section 5. 3 JUDGMENTAL EMBEDDING This section provides the necessary technical details on the implementation of Ferrite’s core con- structs, building up the knowledge required for Section 4 and Section 5. Ferrite, like any other DSL, has to tackle the various technical challenges encountered when embedding a DSL in a host language. In doing so, we take inspiration from the range of embedding techniques developed for Haskell and adjust them to the Rust setting. The lack of higher-kinded types, limited support of recursive types, and presence of weakening, in particular, make this adjustment far from trivial. A more technical contribution of this work is thus to demonstrate how existing Rust features can be combined to emulate the missing features beneficial to DSL embeddings and how to encode cus- tom typing rules in Rust or any similar language. The techniques described in this and subsequent sections also serve as a reference for embedding other DSLs in a host language like Rust. 3.1 Encoding Typing Rules A distinguishing characteristics of Ferrite is its propositions as types approach, yielding a direct correspondence between SILLR notions and their Ferrite encoding. We introduced this correspon- dence in Section 2.1 (see Table 2), next we discuss it in more detail. To this end, let’s consider the typing of value input. We remind the reader of Table 1 in Section 1, which provides a mapping between SILLR and Ferrite session types. Interested reader can find a corresponding mapping on the term level in Table 4 in Appendix A. Γ, a : ; Δ ⊢ :: (T ⊲ R ) Γ ; Δ ⊢ a ← receive_value; :: ⊲ The SILLR right rule T ⊲ R types the expression a ← receive_value; as the session type ⊲ and the continuation as the session type . Ferrite encodes this rule as the function receive_value, parameterized by a value type T ( ), a linear context C (Δ), and an offered session type A. 7
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer fn receive_value < T, C: Context, A: Protocol > ( cont: impl FnOnce(T) -> PartialSession ) -> PartialSession < C, ReceiveValue < T, A > > The function yields a value of type PartialSession< C, ReceiveValue >, i.e. the conclusion of the rule, given a closure of type FnOnce(T) → PartialSession, encapsulating the premise of the rule. The use of a closure reveals the continuation-passing-style of the encoding, where the received value of type T is passed to the continuation closure. The closure implements the FnOnce(T) trait, ensuring that it can only be called once. PartialSession is one of the core constructs of Ferrite that enables the judgmental embedding of SILLR . A Rust value of type PartialSession represents a Ferrite program that guarantees linear usage of session type channels in the linear context C and offers the linear session type A. A PartialSession type in Ferrite thus corresponds to a SILLR typing judgment. The type parameters C and A are constrained to implement the traits Context and Protocol – two other Ferrite constructs representing a linear context and linear session type, respectively: trait Context { ... } trait Protocol { ... } struct PartialSession < C: Context, A: Protocol > { ... } For each SILLR session type, Ferrite defines a corresponding Rust struct that implements the trait Protocol, yielding the listing shown in Table 1. Corresponding implementations for (End) and ⊲ (ReceiveValue) are shown below. When a session type is nested within another session type, such as in the case of ReceiveValue, the constraint to implement Protocol is propagated to the inner session type, requiring A to implement Protocol too: struct End { ... } struct ReceiveValue < T, A > { ... } impl Protocol for End { ... } impl < A: Protocol > for ReceiveValue < T, A > { ... } Whereas Ferrite delegates the handling of the structural context Γ to Rust, it encodes the linear context Δ explicitly. Being affine, the Rust type system permits weakening, a structural property re- jected by linear logic. Ferrite encodes a linear context as a heterogeneous (type-level) list [Kiselyov et al. 2004] of the form HList![A0 , A1 , ..., A −1 ], with all its type elements A implementing Protocol. Internally, the HList! macro desugars the type level list into nested tuples (A0 , (A1 , (..., (A −1 , ())))). The unit type () is used as the empty list (HNil) and the tuple constructor (,) is used as the HCons constructor. The implementation for Context is defined inductively as follows: impl Context for () { ... } impl < A: Protocol, C: Context > Context for ( A, C ) { ... } type Session < A > = PartialSession < (), A >; To represent a closed program, that is a program without any free variables, Ferrite defines a type alias Session for PartialSession, with C restricted to the empty linear context. A complete session type program in Ferrite is thus of type Session and amounts to the SILLR typing derivation proving that the program adheres to the defined protocol. As an illustration of how to program in Ferrite, following shows an example “hello world”-style session type program that receives a string value as input, prints it to the screen, and then terminates: let hello_provider: Session < ReceiveValue < String, End > > = receive_value (| name | { println!("Hello, {}", name); terminate () }); 3.2 Linear Context 3.2.1 Context Lenses. The use of a type-level list for encoding the linear context has the advantage that it allows the context to be of arbitrary length. Its disadvantage is that it imposes an order on the context’s elements, disallowing exchange. To make up for that, we make use of the concept from lenses [Foster et al. 2007] to define a ContextLens trait, which is implemented using type level natural numbers. 8
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual #[derive(Copy)] struct Z; #[derive(Copy)] struct S < N > ( PhantomData ); trait ContextLens < C: Context, A1: Protocol, A2: Protocol > { type Target: Context; ... } The ContextLens trait defines the read and update operations on a linear context, such that given a source context C = HList![..., A , ...], the source element of interest, A at position , can be updated to the target element B to form the target context Target = HList![..., B, ...], with the remaining elements unchanged. We use natural numbers to inductively implement ContextLens at each position in the linear context, such that it satisfies all constraints in the form: N: ContextLens < HList![..., A , ...], A , B, Target = HList![..., B, ...] > The implementation of natural numbers as context lenses is done by first considering the base case, with Z used to access the first element of any non-empty linear context: impl < A1: Protocol, A2: Protocol, C: Context > ContextLens < (A1, C), A1, A2 > for Z { type Target = ( A2, C ); ... } impl < A1: Protocol, A2: Protocol, B: Protocol, C: Context, N: ContextLens < C, A1, A2 > > ContextLens < (B, C), A1, A2 > for S < N > { type Target = ( B, N::Target ); ... } In the inductive case, for any natural number N that implements the context lens for a context HList![A0 , ..., A , ...], it’s successor S then implements the context lens HList![A−1 , A0 , ..., A , ...], with a new element A−1 appended to the front of the linear context. Using context lens, we can now encode the left rule T⊲L shown below, which sends a value to a channel in the linear context that expects to receive a value. Γ ; Δ, : ⊢ :: (T⊲L ) Γ, : ; Δ, : ⊲ ⊢ send_value_to ; :: In Ferrite, T⊲L is implemented as the function send_value_to, which uses a context lens N to send a value of type T to the N-th channel in the linear context C1. This requires the N-th channel to have the session type ReceiveValue. A continuation cont is then given with the linear context C2, which has the N-th channel updated to session type A. fn send_value_to < N, T, C1: Context, C2: Context, A: Protocol, B: Protocol > ( n: N, x: T, cont: PartialSession < C2, B > ) -> PartialSession < C1, B > where N: ContextLens < C1, ReceiveValue < T, A >, A, Target=C2 > 3.2.2 Channel Removal. The current definition of a context lens works well for accessing a chan- nel in a linear context to update its session type. However we have not addressed how channels can be removed from or added to the linear context. These operations are required to implement session termination and higher-order channel constructs such as ⊗ and ⊸. To support channel removal, we introduce a special empty element Empty to denote the absence of a channel at a par- ticular position in the linear context: struct Empty; trait Slot { ... } impl Slot for Empty { ... } impl < A: Protocol > Slot for A { ... } To allow Empty to be present in a linear context, we introduce a new Slot trait and make both Empty and Protocol implement Slot. The original definition of Context is then updated to allow types that implement Slot instead of Protocol. Γ ; Δ ⊢ :: (T1R ) (T1L ) Γ ; · ⊢ terminate; :: Γ ; Δ, : ⊢ wait ; :: Using Empty, it is straightforward to implement rule T1L using a context lens that replaces a channel of session type End with the Empty slot. The function wait shown below does not really 9
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer remove a slot from a linear context, but merely replaces the slot with Empty. The use of Empty is necessary, because we want to preserve the position of channels in a linear context in order for the context lens for a channel to work across continuations. fn wait < C1: Context, C2: Context, A: Protocol, N: ContextLens < C1, End, Empty, Target=C2 > > ( n: N, cont: PartialSession < C2, A > ) -> PartialSession < C1, A > With Empty introduced, an empty linear context may now contain any number of Empty slots, such as HList![Empty, Empty]. We introduce a new EmptyContext trait to abstract over the different forms of empty linear context and provide an inductive definition as its implementation. trait EmptyContext : Context { ... } impl EmptyContext for () { ... } impl < C: EmptyContext > EmptyContext for ( Empty, C ) { ... } Given the empty list () as the base case, the inductive case (Empty, C) is an empty linear context, if C is also an empty linear context. Using the definition of an empty context, the right rule T1R can then be easily encoded as the function terminate shown below: fn terminate < C: EmptyContext > () -> PartialSession < C, End > 3.2.3 Channel Addition. The Ferrite function wait removes a channel from the linear context by replacing it with Empty. The function receive_channel, on the other hand, adds a new channel to the linear context. The SILLR right rule T⊸R for channel input is shown below. It binds the received channel of session type to the channel variable and adds it to the linear context Δ of the continuation. Γ ; Δ, : ⊢ :: (T⊸R ) Γ ; Δ ⊢ ← receive_channel; :: ⊸ To encode T⊸R , the append operation is defined using the AppendContext trait shown below. AppendContext is parameterized by a linear context C, has Context as its super-trait, and an asso- ciated type Appended. If a linear context C1 implements the trait AppendContext, it means that the linear context C2 can be appended to C1, with C3 = C1::Appended being the result of the ap- pend operation. The implementation of AppendContext is defined inductively, with the empty list () implementing the base case and the cons cell (A, C) implementing the inductive case. trait AppendContext < C: Context > : Context { type Appended: Context; ... } impl < C: Context > AppendContext < C > for () { type Appended = C; ... } impl < A: Slot, C1: Context, C2: Context, C3: Context > AppendContext < C2 > for ( A, C1 ) { type Appended = ( A, C3 ); ... } where C1: AppendContext < C2, Appended=C3 > Using AppendContext, a channel B can be appended to the end of a linear context C, if C imple- ments AppendContext. The new linear context after the append operation is then given in the associated type C::Appended. At this point we know that the channel B can be found in C::Appended, but there is not yet any way to access channel B from C::Appended. To provide access, a context lens has first to be generated. We observe that the position of channel B in C::Appended is the same as the length of the original linear context C. In other words, the context lens for channel B in C::Appended can be generated by getting the length of C. trait Context { type Length; ... } impl Context for () { type Length = Z; ... } impl < A: Slot, C: Context > Context for ( A, C ) { type Length = S < C::Length >; ... } In Ferrite, the length operation can be implemented by adding an associated type Length to the Context trait. Then the inductive implementation of Context for () and (A, C) are updated correspondingly. With that, the right rule T⊸R is encoded as follows: 10
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual fn receive_channel < A: Protocol, B: Protocol, C1: Context, C2: Context > ( cont: impl FnOnce ( C1::Length ) -> PartialSession < C2, B > ) -> PartialSession < C1, ReceiveChannel < A, B > > where C1: AppendContext < ( A, () ), Appended=C2 > The function receive_channel is parameterized by a linear context C1 implementing AppendContext to append the session type A to C1. The continuation argument cont is a closure that is given a con- text lens C::Length, and returns a PartialSession having C2 = C1::Appended as its linear context. The function returns a PartialSession with C1 being the linear context and offers the session type ReceiveChannel. The use of receive_channel and send_value_to can be demonstrated with an example hello client shown below. let hello_client: Session < ReceiveChannel < ReceiveValue < String, End >, End > > = receive_channel (| a | { send_value_to ( a, "Alice".to_string(), wait ( a, terminate() ) ) }); The program hello_client is written to communicate with the hello_provider program de- fined earlier in section 3.1. The communication is achieved by having hello_client offer the session type ReceiveChannel. Inside the body, hello_client uses receive_channel to receive a channel of session type ReceiveValue provided by hello_provider. The continuation closure is given an argument a: Z, denoting the context lens generated by receive_channel for accessing the received channel in the linear context. Following that, the context lens a: Z is used for sending a string value, after which hello_client waits for hello_provider to terminate. We note that the type Z of channel a, and thus the channel position within the context, is automatically inferred by Rust and not exposed directly to the user. 3.3 Communication At this point we have defined the necessary constructs to build and type check both hello_provider and hello_client, but the two are separate Ferrite programs that are yet to be linked with each other and be executed. Γ ; Δ1 ⊢ 1 :: Γ ; Δ2, : ⊢ 2 :: (T-fwd) (T-cut) Γ ; : ⊢ forward :: Γ ; Δ1, Δ2 ⊢ ← cut 1 ; 2 :: In SILLR , the cut rule T-cut allows two session type programs to run in parallel, with the channel offered by the first program added to the linear context of the second program. Together with the forward rule T-fwd, we can use cut twice to run both hello_provider and hello_client in parallel, and have a third program that sends the channel offered by hello_provider to hello_client. The program hello_main would have the following pseudo code in SILLR : hello_main : = f ← cut hello_client; a ← cut hello_provider; send_value_to f a; forward f ; To implement cut in Ferrite, we need a way to split a linear context C = Δ1, Δ2 into two sub- contexts C1 = Δ1 and C2 = Δ2 so that they can be passed to the respective continuations. Moreover, since Ferrite programs use context lenses to access channels, the ordering of channels inside C1 and C2 must be preserved. We can preserve the ordering by replacing the corresponding slots with Empty during the splitting. Ferrite defines the SplitContext trait to implement the splitting as follows: enum L {} enum R {} trait SplitContext < C: Context > { type Left: Context; type Right: Context; ... } We first define two marker types L and R with no inhabitant. We then use type level lists consist of elements L and R to implement the SplitContext trait for a given linear context C. The SplitContext implementation contains the associated types Left and Right, representing the contexts C1 and C2 after splitting. As an example, the type HList![L, R, L] would implement 11
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer SplitContext< HList![A1, A2, A3] > for any slot A1, A2 and A3, with the associated type Left being HList![A1, Empty, A3] and Right being HList![Empty, A2, Empty]. We omit the implementation details of SplitContext for brevity. Using SplitContext, the function cut can be implemented as follows: fn cut < XS, C: Context, C1: Context, C2: Context, C3: Context, A: Protocol, B: Protocol > ( cont1: PartialSession, cont2: impl FnOnce(C2::Length) -> PartialSession ) -> PartialSession < C, B > where XS: SplitContext < C, Left=C1, Right=C2 >, C2: AppendContext < HList![A], Appended=C3 > The function cut works by using the heterogeneous list XS that implements SplitContext to split a linear context C into C1 and C2. To pass on the channel A that is offered by cont1 to cont2, cut uses similar technique as receive_channel to append the channel A to the end of C2, resulting in C3. Using cut, we can write hello_main in Ferrite as follows: let hello_main: Session < End > = cut::< HList![] > ( hello_client, move | f | { cut::< HList![R] > ( hello_provider, move | a | { send_channel_to ( f, a, forward ( f ) ) }) }); Due to ambiguous instances for SplitContext, the type parameter XS has to be annotated explic- itly for Rust to know which context a channel should be split into. In the first use of cut, the initial linear context is empty, and thus we call it with the empty list HList![]. We pass hello_client as the first continuation to run in parallel, and assign the channel offered by hello_client as f. In the second use of cut, the linear context would be HList![ReceiveValue], with one channel f. We then request cut to move channel f to the right side using HList![R]. On the left continuation, we pass on hello_provider to run in parallel, and assign the offered channel as a. In the right continuation, we use send_channel_to to send the channel a to f. Finally we forward the continuation of f, which now has the session type End. Although cut provides the primitive way for Ferrite programs to communicate, the use of it is cumbersome and has a lot of boilerplate. For simplicity, Ferrite provides a specialized construct apply_channel that abstracts over the common pattern usage of cut described earlier. apply_channel takes a client program f offering session type ReceiveChannel and a provider program a of- fering session type A, and sends a to f using cut. The use of apply_channel is similar to regular function application, making it more intuitive for programmers to use. fn apply_channel < A: Protocol, B: Protocol > ( f: Session < ReceiveChannel < A, B > >, a: Session < A > ) -> Session < B > 3.4 Program Execution To actually execute a Ferrite program, the program must offer some specific session types. In the simplest case, Ferrite provides the function run_session for running a top-level Ferrite program offering End, with an empty linear context. async fn run_session ( session: Session < End > ) { ... } The function run_session executes the session asynchronously using Rust’s async/await infras- tructure. Internally, the struct PartialSession implements the dynamic semantics of the Fer- rite program, which is only accessible by public functions such as run_session. Ferrite currently uses the tokio [Tokio 2021] run-time for the async execution, as well as the one shot channels from tokio::sync::oneshot to implement the low level communication of Ferrite channels. Since run_session accepts an argument of type Session, this means that programmers must first use cut or apply_channel to fully link partial Ferrite programs with free channel vari- ables, or Ferrite programs that offer session types other than End before they can be executed. This 12
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual restriction ensures that all channels created by a Ferrite program are consumed linearly. For ex- ample, the programs hello_provider and hello_client cannot be executed individually, but the linked program resulting from applying hello_provider to hello_client can be executed: async fn main () { run_session ( apply_channel ( hello_client, hello_provider ) ).await; } We omit the details on the dynamic semantics of Ferrite, as they are matter of programming ef- forts to build an implementation using low level primitives such as Rust channels, at the same time carefully ensuring that the requirements and invariants of session types are satisfied. Interested readers can find more details of the dynamic semantics of Ferrite in Appendix B. 4 RECURSIVE AND SHARED SESSION TYPES Many real world applications, such as web services and instant messaging, are recursive in nature. As a result, it is essential for Ferrite to support recursive session types. In this section, we first report on Rust’s limited support for recursive types and how Ferrite addresses this limitation. Then we discuss how Ferrite encodes shared session types, which are recursive. 4.1 Recursive Session Types Consider a simple example of a counter session type, which sends an infinite stream of integer values, incrementing each by one. To build a Ferrite program that offers such a session type, we may attempt to define the counter session type as follows: type Counter = SendValue < u64, Counter >; If we try to compile our program using the type definition above, we will get a compiler error that says "cycle detected when processing Counter". The problem with the above definition is that it defines a recursive type alias that directly refers back to itself, which is not supported in Rust. Rust imposes various restrictions on the forms of recursive types that can be defined to ensure that the space requirements of data is known at compile-time. To circumvent this restriction, we could use recursive structs. However, if Ferrite relied on recursive struct definitions, end users would have to explicitly wrap each recursive session type in a recursive struct. Such an approach would not be very convenient, so we want to find a way for the end users to define recursive session types using type aliases. One possibility is to perform recursion at the type level [Crary et al. 1999]. In functional languages such as Haskell, it is common to define a data type such as Rec shown below, which turns a non-recursive type f of kind Type → Type into a recursive fixed point type: data Rec (f :: Type -> Type) = Rec (f (Rec f)) In order to implement a similar data structure in Rust, the support for higher-kinded types (HKT) is required. At the time of writing, Rust has limited support for HKT through generic associated types (GAT), which are currently only available in the nightly release. Since GAT are not yet stable, we chose to emulate HKT using defunctionalization [Reynolds 1972; Yallop and White 2014]. A type F can emulate to be of kind Type → Type by implementing the RecApp trait as follows: trait RecApp < X > { type Applied; } type AppRec < F, X > = < F as RecApp < X > >::Applied; struct Rec < F: RecApp < Rec > > { unfold: Box < F::Applied > } The RecApp trait is parameterized by a type parameter X, which serves as the type argument to be applied to. This makes it possible for a Rust type F that implements RecApp to act as if it has kind Type → Type and be "applied" to X. We define a type alias AppRec to refer to the Applied associated type resulting from "applying" F to X via RecApp. Using RecApp, we can now define Rec as a struct parameterized by a type F that implements RecApp< Rec >. The body of Rec contains a boxed value Box. 13
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer An advantage of using RecApp is that we can lift arbitrary types of kind Type and make them behave as if they have kind Type → Type. This means that we can implement RecApp for all Ferrite session types, such that it can be used inside the Rec constructor. For example, the recursive session type Counter would look like Rec. We want the session type SendValue to implement RecApp for all X. To do that we need to pick a placeholder type in the ? position to mark the recursion point of the protocol. For that we choose Z as the placeholder type. The respective implementations of RecApp is as follows: impl < X > RecApp < X > for Z { type Applied = X; } impl < X, T, A, AX > RecApp < X > for SendValue < T, A > where A : RecApp < X, Applied = AX > { type Applied = SendValue < T, AX >; } Inside RecApp, Z simply replaces itself with the type argument X. SendValue delegates the type application of X to A, provided that the continuation session type A also implements RecApp for X. With that, we can define Counter as follows: type Counter = Rec < SendValue < u64, Z > >; The session type Counter is iso-recursive, as the rolled type Rec and the unrolled type SendValue
Ferrite: A Judgmental Embedding of Session Types in Rust ICFP 2021, 2021, Virtual 4.2.1 Shared Session Types in Ferrite. Shared session types are recursive in nature, as they have to offer the same linear critical section to all clients that acquire a shared process. As a result, we can use the same technique that we use for defining recursive session types also for shared session types. The shared session type SharedCounter can be defined in Ferrite as follows: type SharedCounter = LinearToShared < SendValue < u64, Z > >; Compared to linear recursive session types, the main difference is that instead of using Rec, a shared session type is defined using the LinearToShared construct. This corresponds to the ↑SL in SILLR , with the inner type SendValue corresponding to the linear portion of the shared session type. At the point of recursion, the type Z is used in place of ↓SL SharedCounter, which is then unrolled during type application. Type unrolling now works as follows: trait SharedRecApp < X > { type Applied; } trait SharedProtocol { ... } struct SharedToLinear < F > { ... } impl < F > Protocol for SharedToLinear < F > { ... } struct LinearToShared < F: SharedRecApp < SharedToLinear > > { ... } impl < F > SharedProtocol for LinearToShared < F > { ... } The struct LinearToShared is parameterized by a linear session type F that implements the trait SharedRecApp< SharedToLinear >. It uses the SharedRecApp trait instead of the RecApp trait to ensure that the session type is strictly equi-synchronizing [Balzer et al. 2019], requiring an ac- quired session to be released to the same type at which it was previously acquired. Ferrite en- forces this requirement by omitting an implementation of SharedRecApp for End, ruling out invalid shared session types such as LinearToShared< SendValue >. We note that the type ar- gument to F’s SharedRecApp is another struct SharedToLinear, which corresponds to ↓SL in SILLR . The struct SharedToLinear is also parameterized by F, but without the SharedRecApp constraint. Since SharedToLinear and LinearToShared are mutually recursive, the type parameter F alone is sufficient for reconstructing the type LinearToShared from the type SharedToLinear. A new trait SharedProtocol is also defined for identifying shared session types, i.e., LinearToShared. struct SharedChannel { ... } impl Clone for SharedChannel { ... } Once a shared process is started, a shared channel is created to allow multiple clients to ac- cess the shared process through the shared channel. The code above shows the definition of the SharedChannel struct. Unlike linear channels, shared channels follow structural typing, i.e., they can be weakened or contracted. This means that we can delegate the handling of shared channels to Rust, given that SharedChannel implements Rust’s Clone trait to permit contraction. Whereas SILLS provides explicit constructs for sending and receiving shared channels, Ferrite’s shared channels can be sent as regular Rust values using Send/ReceiveValue. On the client side, a SharedChannel serves as an endpoint for interacting with a shared process running in parallel. To start the execution of such a shared process, a corresponding Ferrite pro- gram has to be defined and executed. Similar to PartialSession, we define SharedSession as shown below to represent such a shared Ferrite program. struct SharedSession < S: SharedProtocol > { ... } fn run_shared_session < S: SharedProtocol > ( session: SharedSession ) -> SharedChannel Just as PartialSession encodes linear Ferrite programs without executing them, SharedSession encodes shared Ferrite programs without executing them. Since SharedSession does not imple- ment the Clone trait, the shared Ferrite program is itself affine and cannot be shared. To enable sharing, the shared Ferrite program must first be executed using the run_shared_session. The function run_shared_session takes a shared Ferrite program of type SharedSession and starts running it in the background as a shared process. Then in parallel, the shared channel of type 15
ICFP 2021, 2021, Virtual Ruofei Chen and Stephanie Balzer SharedChannel is returned to the caller, which can then be passed to multiple clients for access- ing the shared process. Below we show a simple example of how a shared session type program can be defined and used by multiple clients: type SharedCounter = LinearToShared < SendValue < u64, Z > >; fn counter_producer ( current_count: u64 ) -> SharedSession < SharedCounter > { accept_shared_session ( move || { send_value ( current_count, detach_shared_session ( counter_producer ( current_count + 1 ) ) )}) } fn counter_client ( counter: SharedChannel < SharedCounter > ) -> Session < End > { acquire_shared_session ( counter, move | chan | { receive_value_from ( chan, move | count | { println!("received count: {}", count); release_shared_session ( chan, terminate () ) }) }) } The recursive function counter_producer creates a SharedSession program that, when executed, offers a shared channel of session type SharedCounter. On the provider side, a shared session is defined using the accept_shared_session construct, with a continuation closure that is called to when a client acquires the shared session and enters the linear critical section of session type SendValue. Inside the closure, the producer uses send_value to send the current count to the client and then uses detach_shared_session to exit the linear criti- cal section. The construct detach_shared_session offers the linear session type SharedToLinear and expects a continuation that offers the shared session type SharedCounter to serve the next client. We generate the continuation by recursively calling the counter_producer function. The function counter_client takes a shared channel of session type SharedCounter and returns a session type program that acquires the shared channel and prints the received count value to the terminal. A linear Ferrite program can acquire a shared session using the acquire_shared_session construct, which accepts a SharedChannel Rust object and adds the acquired linear channel to the linear context. In this case, the continuation closure is given the context lens Z, which provides access to the linear channel of session type SendValue in the first slot of the linear context. It then uses receive_value_from to receive the value sent by the shared provider and then prints the value using println!. On the client side, the linear ses- sion of type SharedToLinear must be released using the release_shared_session construct. After releasing the shared session, other clients will then able to acquire the shared session. async fn main () { let counter1: SharedChannel = run_shared_session ( counter_producer(0) ); let counter2 = counter1.clone(); let child1 = task::spawn ( async move { run_session( counter_client(counter1) ).await; }); let child2 = task::spawn ( async move { run_session( counter_client(counter2) ).await; }); join!(child1, child2).await; } To demonstrate the shared usage of SharedCounter, we have a main function that first initializes the shared producer with an initial count value of 0 and then executes the shared provider in the background using the run_shared_session construct. The returned SharedChannel is then cloned, making the shared counter session now accessible via the aliases counter1 and counter2. It then uses task::spawn to spawn two async tasks that runs the counter_client program twice. A key observation is that multiple Ferrite programs that are executed independently are able to access the same shared producer through a reference to the shared channel. 5 N-ARY CHOICE Session types support internal and external choice, leaving the choice among several options to the provider or client, respectively (see Table 1). When restricted to binary choice, the implementation 16
You can also read