The Emergence of Active APIs

Just as Inversion of Control (IoC) turned traditional programming on its head, modern cloud services are ushering in a new era of application control. Imagine the cloud not just as a passive provider, but an active entity, implementing the same principles of IoC with programmable components that are always on, performing tasks for your application, and driving new interactions. This is the new world of Active APIs—an evolution that resembles traditional IoC frameworks but takes it a step further into the distributed realm, where the cloud actively communicates with our services. Welcome to a journey describing this transformative approach.

Drawing Parallels between Electronics and APIs

In electronics, there are interconnected components forming an elaborate system, performing complex functions. These components consist of passive components, such as resistors and capacitors that consume energy, and active components, like transistors and integrated circuits, that manipulate electricity in the form of voltage or current. This concept resembles distributed systems composed of multiple components deployed as processes and interacting over APIs. And similar to electronics, in software systems we have passive services that react to other stimuli, and active services that can generate events from within. These active APIs can trigger new interactions, orchestrate complex workflows, or alter the flow of data through aggregation and splitting.

‍

Implementing reliable and scalable active services is hard, as they require distributed state management behind the scenes. But in recent years we can see an increase of active services, which now can be consumed as cloud APIs and offload complex implementation and operational responsibilities. This move of active APIs to the cloud extends the idea of cloud beyond just a passive infrastructure provider - it forms an application-cloud that is an active participant in the application control flow. Let’s dive into the details.

Passive APIs as Transparent Proxies and Storage

Passive components in electronics don’t need an external power source to function. They only require the current traveling through the component and use some other property to affect the current or voltage. For example, they can resist its flow, or store energy for later use. However, they cannot control electricity themselves. Passive APIs mimic passive electronic components. Just as passive electronic components operate on incoming current or voltage, passive APIs work on incoming requests. They can alter, direct, monitor, log, trace, or even deny these requests. Passive cloud APIs are typically transparent to the business logic and as such they are used mainly by the operations teams rather than developers. Good example here is the Envoy proxy, which can route and secure requests, or the AWS API Gateway that routes requests to the appropriate service. Another resemblance with passive components is the storage services. Passive APIs, like a database or an event log, can store data for future use, and that is it. Their functionality is consumed through an external trigger, an event or command, and it doesn't produce any stimuli independently.

Active APIs as Programmable Control Flows

Active components in electronic circuits such as amplifiers, transistors and integrated circuits rely on an external power source to control the flow of current, and manipulate the power in a system. They use one electrical signal to control another and are required for the circuit to be considered “electronic.” Similarly, active cloud APIs resemble active electronic components and serve to trigger new interactions, orchestrate complex interactions, or amplify an existing request (enrich, split, aggregate data). For example, an active API can trigger a function based on a predefined setting such as the case with Amazon EventBridge Scheduler or GCP Cloud Scheduler. It may orchestrate complex interactions, such as the Saga pattern as in the case of AWS Step Functions or GCP Workflows. Active APIs are programmable and configurable by developers who are aware of the business logic of the application, and they can for example aggregate multiple requests into a single one (aggregator pattern), split a request into multiple ones (splitter pattern), or enrich a request with supplementary data from another endpoint (content enricher pattern). Such an API can poll an external data source (polling consumer pattern) and trigger events as in the case of Apache Kafka Connect or Dapr bindings. By connecting your application to such Active APIs, developers can focus on implementing the business logic and offload complex stateful distributed system concerns to other services.

‍

To offer such functionalities, typically, an active component needs to have state, its own processing logic that is customizable, and run as a logically or physically separate process. The Dapr’s sidecar architecture is an excellent embodiment of the active API concept. Dapr not only offers passive APIs, which are called by the application when interacting with state stores, for example, but also active APIs. Active APIs in Dapr include workflows that coordinate interactions between services, input bindings that poll external endpoints and pass the data to the app, cron triggers that call the app periodically, configuration change updates, etc. In addition, Dapr’s sidecar architecture aligns and scopes APIs to each microservice, implementing them in a way that's consistent with the unique responsibilities of each service. This structure ensures that the microservice and its active APIs remain cohesive, while offloading the implementation responsibilities from the developer by reusing complex stateful logic in a dedicated Dapr runtime. That said, there's no hard requirement for Dapr to be locally deployed as a sidecar. Instead, Dapr can be deployed as a remote, serverless sidecar, that binds with the application as an active API, making it a per-microservice serverless sidecar. Looked at from this perspective, active APIs represent an evolution of the so-called Multi-runtime architecture. In this evolution, the mecha runtime addressing infrastructure connectivity responsibilities shifts from a local sidecar to the cloud in the form of active cloud APIs.

The Cloud as a Distributed Inversion of Control Framework

In the world of object-oriented programming, a core design principle has been the Inversion of Control (IoC) - a concept where the control flow of a program is inverted: instead of the developer controlling the flow of application by calling methods, the framework calls the application methods. An IoC framework ties together separate functionalities and guides an application's execution. This phenomenon is also known as the "Hollywood Principle" - "Don't call us, we'll call you" - has been a foundation of many application architectures, promoting modularity, extensibility and testability.

‍

Active APIs extend the IoC concept beyond the boundaries of a single process and application into the cloud. Cloud services, through active APIs, can now trigger and control application logic, effectively calling application endpoints when required. Much like the traditional IoC, where a framework takes control of flow, the cloud assumes this responsibility. In this model, we deploy services and functions with remotely accessible endpoints rather than classes. Then bind the application's endpoints into the cloud's active APIs, much like injecting custom behavior into a traditional IoC framework. The cloud calls endpoints of our application as needed, directing the execution flow across multiple services and functions. In essence, the cloud is no longer a passive resource provider; it's becoming an active participant in the application’s control flow, adopting a distributed IoC framework role.

In conclusion

Passive APIs, similar to remote libraries, respond to stimuli, execute a task, and then return control to the client. They primarily address operational needs and offer state persistence, acting transparently to the application logic. Most of today's developer-oriented cloud services are similar to these passive APIs, mainly providing critical storage or networking services. However, they often necessitate complex operational overhead from developers, who must implement control logic within the application.

Active APIs offer a contrasting approach. Developers configure these APIs to modify requests, generate new events, and manage control logic. Their stateful and programmable nature makes them similar to distributed Inversion of Control (IoC) frameworks. Here, the active cloud APIs initiate communication with our application, a stark shift from traditional paradigms. This approach is akin to how the IoC principle delegated control to a framework from specific code modules, encapsulating complex logic and improving application flexibility, scalability, and reliability. By extending the IoC principle to the cloud level, applications can operate more decentrally and resiliently. This shift follows the "Hollywood Principle"—don't call us; we'll call you—on a grand scale, transforming the cloud from a mere infrastructure provider to an active participant in application logic.