The Bifurcation of Application Networking

In my talk “Where Has My Circuit Breaker Gone” at KubeCon Chicago, I explored the evolving landscape of distributed application interactions. The focus was on the shift of features like circuit breakers, connectors, filters, event-driven interactions, and workflows. These features are increasingly moving away from the application stack. Here is a summary of the key trends and takeaways from the presentation.

Fragmentation of Application Architectures

We are witnessing an acceleration of application and infrastructure fragmentation. Moving from a world of on-premise monolithic applications, we now see a landscape of functions, microservices, and not-so-micro services transitioning to hybrid cloud. This shift from larger to smaller, modular units is accelerated by technologies like Kubernetes and Knative, which abstract infrastructure complexities and allow operations teams to deploy, monitor, and operate diverse distributed applications at scale.

We have reached a point where distributed applications are made up of different sizes and shapes, written in different languages, and running on multiple clouds. These applications interact with each other using both synchronous and asynchronous patterns, and consume purpose-built backing cloud services over a variety of protocols and APIs. In this world of fragmented applications and cloud services, the cloud-native movement abstracted the compute infrastructure and automated the operational concerns with polyglot APIs, but it left developers' concerns unaddressed.

The overcrowded room of attendees at the AppDeveloperCon track of KubeCon signals a clear demand for change from developers. Projects such as Dapr, Knative, OpenFeature, and OpenFGA  are stepping up to meet this need. These initiatives embrace cloud-native principles to provide APIs and abstractions that transcend language and cloud boundaries. In this post, we'll explore how traditional middleware functions are shifting away from being embedded within applications. They are transforming into polyglot, universally accessible cloud-native developer capabilities. This transformation is not unidirectional; while some functionalities are sinking deeper into the operating system, others are rising to become standard developer APIs. Next, we'll delve into different types of application interactions and their evolution.

Application Interaction Types

To effectively analyze the broad spectrum of application interactions, we can categorize them into four main groups. First, Lifecycle interactions focus on the application’s engagement with the compute platform, including health check probes, metrics scraping, and log collection. Second, Transparent interactions encompass inter-service communication, covering service discovery, load balancing, and resilience features. Third, Explicit interactions involve connecting to external services or APIs, including protocol conversion, message transformation, and other message modifications, which are not transparent to the application and developers. Finally, Stateful interactions involve data persistence in a medium between interacting services.

‍

We will now delve into each of these categories to understand their evolution in the context of adopting cloud-native principles.

Lifecycle Interactions

In today's cloud-native world, application lifecycle management is key to the processes of deploying, managing, and scaling applications. Platforms like Kubernetes have revolutionized this area by automating the selection of the appropriate node for deployment, starting applications, and dynamically scaling them in response to workload variations, along with health monitoring and healing.

These interactions have become well adopted and standardized, evident in health-check APIs for applications and Prometheus metrics endpoints, which are automatically handled by application frameworks. The outcome is a considerable benefit for developers, enabling them to concentrate more on crafting business logic without being weighed down by operational complexities.

Transparent Interactions

Synchronous service-to-service interactions in today's distributed systems have grown in complexity. Service mesh technology has been pivotal in abstracting many of these runtime complexities. It transparently manages aspects such as service discovery and load balancing, and implements advanced traffic management strategies like A/B testing and canary releases. Besides networking, service meshes also introduce resilience features like circuit breakers and timeouts, which enhance the system's capacity to manage failures effectively. They further bolster inter-service communication with security features like encryption and access control. A key characteristic of these networking capabilities is their transparency to both the application and developers. These features can be deployed at runtime by operations teams without necessitating any changes in the application code, hence the name “transparent interactions”.

Let’s briefly explore how service-mesh-like technologies are transforming the transparent networking capabilities space and the direction of evolution. This journey began with the implementation of libraries like Netflix Hystrix, which provided patterns for resilience and fault tolerance directly within application code. This approach was limited to a single language and Netflix soon packaged these capabilities into a standalone process called Netflix Prana, opening it up to more languages. The landscape then shifted towards more decoupled approaches, exemplified by the introduction of Linkerd, pioneering the sidecar proxy model on Kubernetes. This model was further experimented by Istio, which leveraged a node-level proxy with Istio Ambient. The latest advancement in this space is the integration of service mesh capabilities directly into the operating system using eBPF technology, as seen in projects like Cilium.

This diagram offers a taxonomy and highlights the evolution of transparent networking concerns, illustrating their journey from application-specific libraries to separate processes, and ultimately, their integration into kernel hooks. This progression, however, is not linear and doesn't signify a final state; instead, what we observe now is a varied blend of hybrid deployments. Despite the intricacies, one trend stands out: transparent networking capabilities are progressively moving deeper into the computing infrastructure. This movement allows for more advanced and fluid control over service interactions, without imposing additional complexity onto the application code.

Non-transparent Interactions

Interacting with third-party endpoints is a common requirement for modern applications. A key distinction of these interactions from transparent interactions is the necessity for some level of modification in the message shape or routing logic, for which the application and developers are responsible. Traditionally, this involved tasks like protocol conversion, message transformation, and content-based routing, typically managed within the application. Such functionalities have been handled by projects like Apache Camel and Spring Integration in the Java ecosystem, along with comparable libraries in other programming languages.

The prevailing trend for these capabilities is their expansion to support more languages and their movement away from being tied directly to application deployment. Frameworks such as Dapr, Knative, and even Kafka Connect exemplify this shift, offering language-agnostic methods for handling these tasks. This evolution greatly streamlines the process of connecting to various third-party services, fostering interoperability and diminishing the need for custom coding in each instance of external interaction.

Stateful Interactions

Unlike the previous interaction types which occur synchronously between two systems, where the message is always in one system or the other, stateful interactions are different. In stateful interactions, there is a medium within the network that retains the request from Service A before transmitting it to Service B. These interactions are primarily concerned with data persistence and state management in distributed systems. Common examples include pub/sub message brokers and workflow engines enabling stateful orchestration patterns. These systems facilitate asynchronous communication and can influence it through patterns like the idempotent filter, among others.

Historically, stateful interactions were confined to specific languages and frameworks. Now, they are embracing cloud-native principles. Technologies like workflow engines, message brokers, and state stores are expanding their accessibility through HTTP and gRPC APIs, or via SDKs for various languages. An exemplary case is the Dapr project, which provides uniform interfaces for different messaging systems through its PubSub API, and for datastores through its State API. It also supports patterns like the transactional outbox, independent of the underlying datastore or message broker. This evolution not only simplifies the coordination of stateful services but also enhances the accessibility and efficiency of these capabilities across diverse programming languages and application architectures.

Summary

The shift from larger to smaller application deployment units is influencing not only compute aspects but also application interactions. Within this domain, two concurrent trends are moving in opposite directions, collectively reshaping the trajectory of application networking capabilities.

‍Transparent application interactions, encompassing both lifecycle and networking aspects, are progressively descending towards the operating system, becoming integral to the lower levels of the technology stack and, ultimately, the OSI network model. Responsibilities such as load balancing, service discovery, and traffic management are increasingly being undertaken by the underlying platforms and infrastructure, rather than being embedded directly in the application code. This trend is illustrated by service meshes, which automate resilience and security features in a transparent manner. This shift significantly reduces the operational complexity for developers, who no longer need to incorporate intricate networking logic within their applications. They can depend on the platform to manage these aspects, resulting in cleaner, more focused application code.

‍

Explicit application interactions, encompassing both stateless and stateful elements, are the aspects of networking in which developers are actively involved and aware. These interactions are coalescing into standardized, accessible APIs. Networking functions that directly impact business logic, such as third-party API integrations, state management, and data persistence, are transitioning from being embedded within application code to external APIs. Frameworks like Dapr are leading this shift, offering language-agnostic APIs for a broad spectrum of networking functions. This unification allows developers to engage with complex networking functions through streamlined, universal APIs, accelerating the learning process and enabling portability. Additionally, it nurtures a more inclusive environment where developers of various programming languages can equally access these advanced networking capabilities.

These changes mark a substantial shift towards more efficient, scalable, and manageable distributed application development. As transparent networking responsibilities descend deeper into the infrastructure and explicit networking consolidates into accessible APIs, we observe a bifurcation of networking concerns. Recognizing and understanding this transition is vital for fully leveraging cloud-native technologies and for the creation of robust, scalable distributed systems in the future. Interested in exploring Dapr further? Join the Dapr Discord server to engage with a dynamic community of developers. Connect with us to discover how Diagrid Conductor assists organizations in confidently operating Dapr in production environments.