Spring Kafka Streams using Spring Cloud Streams End to End microservices

Event Driven web domain crawler service using Kafka Streams, Spring Cloud Streams with Kafka Binder and Java Functional Interfaces.

Microservices

• domain-crawler - uses Spring Kafka with KRaft consensus algorithm
• domain-processor - uses Spring Cloud Stream with Kafka Streams binder
• domain-service - uses Spring Cloud Stream with Kafka Streams binder

Architecture

Getting Started

Prerequisites

• Java (17 or above)
• Maven (for building the project)
• Docker
• Apache Kafka

Running the Application

1. Clone this repository:

   git clone https://github.com/bruce-mig/web-domain-crawler-kafka.git
   cd web-domain-crawler-kafka

2. Run Apache Kafka with Docker

   docker compose up -d

3. Build and run the microservices:

   # domain crawler
   cd domain-crawler
   mvn spring-boot:run
   
   # domain processor
   cd domain-processor
   mvn spring-boot:run
   
   # domain service
   cd domain-service
   mvn spring-boot:run

Endpoint

• http://localhost:8080/domain/lookup/facebook - to pull all facebook related web domain names