In lectures regarding object-oriented programming, I sometimes encounter students questioning the use of Java’s interfaces. Why do we need to go through the hassle of defining a class as well as an interface? Once they are convinced (or conditioned) to use them, the questions do not end: students often wonder when interfaces are and are not necessary and how they can contribute to well-designed software.

In this post, we will uncover the power of protocols, a more general term for interfaces, and dive into some common implementations. Along the way, we will discuss why interfaces are useful and how they are commonly used to shape the design and architecture of (object-oriented) software.

Interfaces

Many programmers familiar to statically typed object-oriented languages like Java and C# are familiar with interface as a language construct. However, the general public might think of a user interface when asked to describe an interface: a collection of screens and input/output devices to operate some piece of technology.

To some extent, these ideas overlap. A user interface sits in between the user and the device allowing ways of interacting with it, whereas an interface in the object-oriented sense prescribes ways of interacting with the classes that implement it. However, as we will see, interfaces have some extra strengths.

Sometimes, the term interface refers to a program’s, module’s or class' Application Programming Interface (API): the set of method signatures that describe the ways of interacting with said subject. Note that in this definition the interface is not necessarily explicitly enforced.

In other cases, an interface can be seen as the thing that separates the public from the private. A gateway or facade to an encapsulated subsystem.

Protocols

With regards to programming, an interface is a language-specific construct. The more generalized construct is often referred to as protocol, although some languages like Swift co-opted that term for the underlying constructs as well.

The term protocol is in itself ambiguous. It is commonly used to describe a set of rules governing data transmission or exchange between devices (think: TCP/IP), while in legal terms it is also a system of rules regarding procedures, affairs of state or diplomatic occassions. In a sense, these definitions do match the intent and effect of what protocols in a programming sense are used for. Therefore, we will speak of protocols for referring to the construct, offered by a language, allowing the creation of a higher-level abstraction that can prescribe and restrict the means of interaction between classes, structs or objects and is enforced by a compiler or interpreter.

A typical scenario

Imagine the following scenario. We are working on an application that keeps track of all the books we are want to read, are reading and have read. Within the context of this tracking system, we probably have some domain entity for a Book, containing a book identifier (i.e. the ISBN or some UUID) and the status we give it. We create a BookService that is responsible for all operations regarding the administration of these books. We could split this out in separate command and query handlers, but for this example a general (application) service will do.

During the runtime of our system, we want to keep the status of each book in memory. For this, we can use a collection. But, for reasons that will be clear soon, we will create a service. Let’s call it an InMemoryBookStorage for now. As it will have the ability to set/get a book’s status by its ID and remove it using its ID, it might use some sort of map as its internal, encapsulated datastructure.

We quickly realize that the in-memory storage only stores our status during the runtime of our system. There is a good chance we need to store our book statuses longer. We need persistence. We want to design it right. In the foreseeable future, we might use a database, a distributed storage or a web API, but for now persistence using the local file system suffices. How would we go about this?

We need to create a separation between the kind of functionality we need (the abstraction) and the way it is provided (the implementation). This is one of the things protocols are used for.

Protocols in Programming Languages

Different languages implement protocols differently. Whereas some languages have no support for explicitly introducing an abstraction that offers no externally visible functionality, other languages have extremely expressive ways of separately defining (abstract) types and the possible ways they can be operated upon.

Python’s duck typing

In dynamically typed languages like JavaScript and Python, one does not depend on an explicitly defined abstract protocol. Instead, a class or function depends on an implicit protocol, although not strictly enforced: we assume the methods and attributes depended upon will be present on the injected object during runtime. This is called duck-typing: “If it looks like a duck and quacks like a duck it must be a duck.”

Python’s documentation emphasizes that duck-typing is a programming style “which does not look at an object’s type to determine if it has the right interface; (…) the method or attribute is simply called or used (…).”

In our example, this means we replace InMemoryBookStorage with a FileBookStorage that has the same methods invoked by BookService as were present on the InMemoryBookStorage. For every other storage mechanism we need to support, we will introduce new classes that all have the same method signatures.

Although we do not enforce the code will work correctly before runtime, we will get a runtime error when a method or attribute is missing from the injected object when calling it. Duck-typing is common with a style of programming in which one assumes the input is valid and one does not do a lot of checks on beforehand. Exceptions are dealt with once things go wrong. This style, common to the realm of Python, is referred to as “Easier to ask for forgiveness than permission” (EAFP) and is contrasted with “Look before you leap” (LBYL) in which pre-conditions are checked extensively.

Flexibility

The major benefit of duck-typing is its flexibility. We can depend on a concrete implementation. If we need to use some other service object, we need to make sure the method signatures match the current implementation and the required attributes are present. This means we can even inject third-party objects as long as their attributes and methods are similar to the one we depend upon – or we need to use a wrapper object that does conform to that interface (i.e. an adapter).

This flexibility has a large drawback, especially prevalent in larger projects: compatibility is only checked during runtime. This is problematic because we implicitly couple implementations. The implementations all need the same method signatures. This is prone to errors as a system grows and/or knowledge about the system fades. Furthermore, when that inevitable moment comes that extra functionality is needed we expect more of our implementations. We must then not forget to add the needed extra methods to all of our implementations.

Another drawback is that it is not directly clear from the code itself what the significance of the implementation is. As we are always depending on concrete implementations, we cannot easily tell whether we need to depend on some storage, and any storage will do, or it has to be exactly a file storage. This line of thinking affects the software design process, as there will typically be more focus on concrete implementation details and less focus on the core abstractions that shape the application.

More guarantees

In Python, one way of countering these drawback is to use some form of static type-checking like mypy. This is exactly what Dropbox did to make their 4 million lines of Python code more maintainable.

Besides static typing, investing in a test suite that verifies runtime behaviors could be beneficial. Especially useful would be integration tests testing whether a certain dependency is fulfilled correctly: does the injected object conform to the object needed? Alternatively, conformance tests can be implemented as runtime checks on the objects themselves, but this would be less in line with the practice of dynamically typed languages that favor EAFP over LBYL.

Finally, one could complement duck-typing with Python’s built-in Abstract Base Classes (ABCs):

(…)

ABCs are simply Python classes that are added into an object’s inheritance tree to signal certain features of that object to an external inspector.

(…)

[T]he ABCs define a minimal set of methods that establish the characteristic behavior of the type. Code that discriminates objects based on their ABC type can trust that those methods will always be present.

(…)

— PEP 3119 -- Introducing Abstract Base Classes

Java’s nominal types

Java is a statically typed class-based object-oriented programming language, comparable to C#. Classes are used to group state and behaviour in order to fulfill some responsibility. Objects are concrete instances of classes. An object or class encapsulates its state in its properties, hiding it from the outside world through visibility modifiers. Methods are the means of interacting with the object. These interactions can be classified as commands or queries. In short, commands can modify state or affect some external system while queries are non-mutating questions regarding the internal state of the object.

There are two types of abstract types in Java: abstract classes and interfaces. These are types that cannot be instantiated directly.

Classes can be made abstract. This means they need to be inherited first: a subclass needs to specify that it extends a certain superclass. This concept is also known as ‘implementation inheritance’. Abstract classes allow some methods to be predefined on the abstract superclass, while others need to be implemented on the subclass.

Interfaces are somewhat comparable to an abstract class in which every method needs to be implemented by its subclass. This is sometimes refered to as ‘interface inheritance’. In Java, this means the class needs to signify it implements this interface. An important difference between abstract classes and interfaces is that a single class can implement multiple interfaces, but cannot inherit from multiple (abstract) classes.

By extending an abstract class or implementing an interface, the subclass conveys that, although it has its own concrete, specific type, it conforms to the protocol of its more abstract, general parent type.

Using interfaces or abstract classes as our higher abstraction we can apply the Protected Variation pattern from GRASP by identifying points of predicted variation and creating a stable interface around them. This pattern was first described by Alistair Cockburn in Pattern Languages of Program Design 2, a book by John Vlissides and others from 1996. This is closely related what is referred to as Encapsulate what Varies: isolate variance in their own classes, separated by their own (implied or explicit) interface. This reduces the amount that needs to change in the future.

In our example, this means we let our InMemoryBookStorage be an implementation of a more abstract BookStorage. Let that be an interface. This way, we can vary the implementation of BookStorage through the powers of polymorphism: one (abstract) concept can have many forms. This approach satisfies Meyer’s Open-Closed Principle, which has also been codified into SOLID:

Open-Closed Principle (OCP)

Modules should be both open and closed.

A module is said to be open if it is still available for extension. (…)

A module is said to be closed if it is available for use by other modules. (…)

— Meyer, B., Object-Oriented Software Construction (1988) , p. 57

In our case, we allowed extension through the creation of new implementations but have restricted these implementations by defining an (abstract) interface. We also shield the concrete implementation from outside changes through the power of encapsulation. This interpretation of OCP is more in line with the way it is described in Martin’s SOLID principles.

Our BookService needs some sort of BookStorage, but it does not care about which exact implementation we inject into it. Like the InMemoryBookStorage, our FileBookStorage will also be an implementation of BookStorage. During runtime, we can give our FileBookStorage to our BookService’s constructor as a dependency to fulfill its need for a BookStorage.

In other words, we take advantage of SOLID’s Dependency Inversion Principle:

Dependency Inversion Principle (DIP)

High-level modules should not depend on low-level modules. Both should depend on abstractions.

Abstractions should not depend on details. Details should depend on abstractions.

— Martin, R. C., “The Dependency Inversion Principle”, C++ Report (May 1996), p. 6

Indeed, as the Gang of Four (Gamma e.a.) said in their Design Patterns book of 1996, we need to “program to an interface, not an implementation.” Clients should depend on the protocols of a certain application, module or class, not on its internal mechanisms. With Java’s interfaces, a client can specify the methods it expects a certain dependency to have.

The flawed contract metaphor

A common way of teaching interfaces in Java and similar languages is to think of them as a contract between the class and the outside world. I disagree. It might be my past self talking, but I don’t think this metaphor clarifies a lot and it is dependent on the (legal) culture in question.

Generally speaking, a contract is thought of as some form of binding agreement between two or more parties constructing mutual obligations between them: i.e. money in exchange for the sale (and delivery) of some good or service. An interface is not drafted by the parties it applies to, nor does it create mutual obligations.

An interface — within the universe of the software at hand — is created by omnipotent beings: the development team. It is a directive regarding the classification of and interaction with a certain subject within that universe: if something implements an interface, they qualify as a certain higher-order type within our system, but in order to qualify they are required to exactly implement certain methods. Interfaces are therefore more like laws written by programmers than contracts between classes. In a country where the object-oriented programmers rule, interfaces govern the legal position of the country’s subjects: their objects. Interfaces specify the behaviours a certain class needs to ensure before it is allowed to call itself of the same type of that interface.

Nominal type systems

Interfaces are a part of a lot of languages that have a nominal type system. In these languages, two instances are compatible if they are declared as being of the same type. Nominal subtyping entails that some type can be a subtype of another if it is explicitly declared to be so.

The biggest benefit of nominal types is their explicitness: it is pretty clear which class implements which interface and extends which (abstract) class. However, a tree of (inherited) classes can get difficult to follow. This is one of the reasons one is advised to “favor composition over inheritance” (Gang of Four 1994, p. 20). Most problems requiring inheritance can be solved using composition, trading duplication for decoupling. Luckily, Java and similar languages allow restriction of extension (or even mutation) through the final keyword.

Like stakes for a tree, Java’s interfaces introduce a guide rail through which our application can grow and evolve. We can (and should) depend on interfaces that can be filled in and vary at another time: pluggability through polymorphism. This protects the structure of our application at the cost of flexibility.

This loss of flexibility is the biggest drawback of nominal types. While other classes can depend on interfaces in their methods, a class specifies which interface or interfaces it implements. This means a class cannot be made to implement an interface after-the-fact. Third-party classes cannot be made to implement our interfaces, even though their method signatures match our interface definition. One solution for this is the Adapter pattern: create a wrapper class around the third-party class that implements the interface we want and passes the method calls to the underlying class.

Reduced flexibility has another drawback: a change in the interface means a change in every single implementor. One way to mitigate this problem is to follow the Interface Principle, the ‘I’ in SOLID:

Interface Segregation Principle (ISP)

Clients should not be forced to depend upon interfaces that they do not use.

— Martin, R. C., “The Interface Segregation Principle”, C++ Report (May 1996), p. 5

Generally speaking, this means interfaces should be kept small and coherent.

Rust’s traits

In class-based programming languages like Java, state and behaviours are defined to be grouped as a single logical component, the class. The object, the instantiation of a class, holds a certain state which can be modified by other objects through its methods over time. Although object-oriented programming is possible, Rust is not a class-based programming language. In Rust, state and behaviours are represented separately.

Structures (struct) are the primary way to define custom, complex data types in Rust. This way, primitive data types can be combined into a single component. As you would expect, Functions can then operate on structs by passing them in as arguments and can mutate the existing struct, return other structs or primitive types or produce some kind of side effect.

In addition to functions, methods can be added to structs by defining functions (fn) for a specific struct implementation (impl). Defining methods done separately from defining a struct and it is allowed to have multiple impl blocks for the same struct. This makes it possible to add methods to any struct — even those we do not control.

In Rust, protocols are implemented as traits. Traits can be used to define shared behaviour in an abstract way, which can be implemented as methods for a certain struct. We can let certain struct attributes or method parameters depend on any type that implements a certain trait or even a combination of traits. The separation of state and behaviour makes it possible to add behaviours to structs that adhere to a certain trait after-the-fact, so that even third-party structs can be made to fit the mould of a desired trait. This is referred to as ad-hoc polymorphism: the implementation of a method or functions depends on the type of the arguments.

Interestingly, one can even depend on an abstraction that needs to conform to several traits at the same time by using multiple trait bounds, effectively allowing depending on sum types. Traits can therefore be used to allow for constrain-based generic programming, like Haskell’s top-notch typeclasses. In this style of programming, which also lies at the heart of Swift’s protocol-oriented programming you start with the core abstraction first and explicitly define the context under which it is applicable by constraining its type. This can lead to small, cohesive abstractions that can support multiple cases and can be combined into larger abstractions.

In our example, we could define the abstract BookStorage as a trait, requiring methods for reading and writing Books. Interestingly, we could start with a trait for only reading books and another for writing books and defining our BookStorage as a combination of the two. We could even use a super trait and/or multiple trait bounds for that. InMemoryBookStorage and FileBookStorage can then be structs containing the data and references required for storing Books. The BookStorage trait can then be implemented for these specific structs by defining the required methods for them.

This closely resembles the design of the Java example, but is more flexible because of the versatile nature of Rust’s type system. Methods implementing the neccessary traits can be added on existing storage solutions and traits can be used for creating generic storage implementations. This means design patterns like the adapter pattern will be less useful in Rust, although we may still want to adapt a single method of the code we cannot control to a more desirable protocol.

Go’s structural typing

Like Rust, Go’s primary data container is a struct and methods can be defined on structs separately. However, unlike Rust, Go does not have anything generic like Rust’s traits that can be implemented for (the methods of) a certain struct.

Go has interfaces. A client’s method or a general function can depend on an interface. Structs, however, cannot signify that it adheres to a certain interface. There is no implements keyword. In Go, interfaces are implemented implicitly: if a struct contains the required methods that conform to the required signatures, it is said to conform to a certain interface. This means that interfaces carry type information, rather than objects. This is verified statically by the compiler, which has lead some people to refer to Go’s form of typing to “static duck-typing”. A more common way of referring to this way of typing is “structural typing”, because the structure of a certain type determines whether it conforms to a certain interface or not. This allows for the definition of adding implementations or interfaces after-the-fact, even when dealing with third-party code.

Comparable to the “classical” notion of Interface Segregation, it is considered best practice in Go to have small, cohesive interfaces. The Go Proverbs read: “The bigger the interface, the weaker the abstraction”. Indeed, large interfaces require a lot of methods to be present on the implementing type. Dealing with structural typing, one needs to be mindful of what is required of a collaborating dependency.

In Go, this resulted in a lot of specific interfaces, often implementing only a single method. The io.Writer interface, for instance, is defined as anything that writes a sequence of bytes to some stream, while the io.Reader interface does the opposite.

Any struct that has both Write() and Read() defined can be said to implement both the Writer and the Reader interface. However, one cannot explicitly typehint against both interfaces in a client struct or function! There is no such thing like Rust’s multiple trait bounds. In Go, like in Java, if we want to typehint against a combined interface, we need to wrap the interfaces in another (or declare them separately from existing interfaces). While one could use inheritance in Java, this is not the case in Go. We can use embedding for that.

In our example, instead of dealing with raw bytes, we would deal with our more abstract domain concepts when defining our BookStorage interface. For flexibility reasons (and to honor one of the Go Proverbs), it might be a good idea to first define a BookReader and a BookWriter and later, if it makes sense, embed them in a new BookStorage interface.

In conclusion

In this post, we explored the power of protocols by looking at some common object-oriented design principles that aid maintainability. These principles exploit object-oriented features like abstraction, polymorphism and encapsulation in a way that a stable foundation emerges. Protocols are instrumental for encapsulating what varies, being open for extension yet closed for modification and depending on abstractions instead of implementation details.

Furthermore, we concluded that duck-typing is extremely flexible, but lacks clarity and enforcement in larger systems. Nominal typing allows explicitly enforced protocols through abstract types (i.e. interfaces), but are not always flexible enough over time. Rust’s traits combat this by allowing implementation of methods on structs after-the-fact. In Go, flexibility is achieved through structural typing: types do not need to declare that they conform to a certain interface, the compiler checks whether a given class adheres to the interface that is required by the dependee.

Protocols are incredibly useful for designing applications that have a stable foundation, but can evolve over time.