Migrating from Kafka to Pulsar. How FICO harnesses Dapr to build high scale messaging.

FICO shares their experience migrating from Apache Kafka to Apache Pulsar using Dapr to build high-scale messaging services.

From Credit Scoring to Platform Ecosystem

FICO transitioned from credit scoring to a platform-first approach, adopting StreamNative Pulsar for decoupled architecture that enhances flexibility and scalability.

Building Decoupled Architecture

FICO developed event services guided by API-first design, separation of concerns, and declarative CI/CD models. Selection criteria for event brokers included security (pluggable authentication, encrypted payloads), scalability (multi-cloud support, low-latency messaging), and flexibility (schema management, multi-tenancy).

Dapr's Role

Dapr serves as an abstraction layer ensuring technology agnosticism. The distributed application runtime offers lightweight building blocks compatible with Kubernetes, including gRPC and HTTP REST APIs.

Technical Challenges Addressed

Key challenges included schema enforcement, OIDC authentication, diverse subscription types, message duplication in Apache Pulsar, and resource consumption optimization.

Strategic Lessons

1. Architectural agility enables scalability and flexibility
2. Technology abstraction allows adaptation to future shifts
3. Community collaboration accelerates problem resolution