back to work
Distributed · Kafka
EventTracer
Personal Project · A playground to visualize Kafka
A distributed order-processing simulator that visualizes event-driven architecture in real time — six choreographed microservices communicate purely through Kafka events, with every step streamed live to the browser as an interactive saga timeline.
NestJSTypeScriptNode.jsApache KafkaKRaftPostgreSQLWebSocketNext.jsTailwind CSSDocker ComposeHetzner
Responsibilities
- Architected and shipped an event-driven microservice system end-to-end — six NestJS services choreographing a distributed saga over Kafka (KRaft mode, no Zookeeper), with a live WebSocket visualization of every event.
- Designed a pure choreography saga (Order → Payment → Shipping → Notification, with a Refund compensation path) in which services never call each other synchronously — they react only to events, so there is no central orchestrator.
- Implemented the transactional outbox pattern in every producing service, so events are written to the database in the same transaction as the state change and relayed to Kafka reliably — no lost or phantom events on crash.
- Built a dedicated Event Monitor service that consumes all topics and streams them to the UI over WebSocket, keeping observability fully decoupled from the domain services.
- Made every consumer idempotent against at-least-once delivery, deduping on eventId / idempotency key so replayed or duplicate events are safe no-ops.
- Deployed the full stack (Kafka, Postgres, all services, Next.js UI) via Docker Compose on a Hetzner instance behind a reverse proxy, with published ports bound to localhost only.
Key Actions & Decisions
- Enforced a strict command/event split: the browser only ever POSTs commands to a single API Gateway and receives events back over WebSocket — the browser never publishes to Kafka and domain services never push to the UI, keeping the trust boundary clean.
- Chose choreography over an orchestrator to make the event-driven coupling the visible subject of the project — the UI shows services independently reacting to a shared event log.
- Centralized all event contracts (envelope + topic names) in a shared package so no service can redefine a schema or topic string — every event carries eventId, eventType, occurredAt, correlationId, version, and payload.
- Used the correlationId (the saga/order id) as the grouping key so the UI can assemble a coherent per-run timeline out of independently produced events.
- Seeded all simulated outcomes (payment success/failure, latency) deterministically so demo scenarios — including the failed-payment refund compensation — are exactly repeatable.
- Ran Kafka in KRaft mode to drop the Zookeeper dependency, simplifying the local and single-node production footprint.
Result
Deployed and live. Issuing an order command drives a full choreographed saga across six services and renders each event on a real-time timeline grouped by correlation id. Demonstrates the transactional outbox, idempotent consumers, DLQ handling, and a failed-payment → refund compensation flow — visualized end-to-end in the browser.