What is Kafka Connect? At first glance, it wasn’t obvious why do we need Kafka Connect. As the name suggests, it seems to me that it is just another feature or tool like Kafka Bridge. Honestly, I can’t tell what are the differences that they are going to bring from the name itself, it…

Kafka Authentication with SCRAM SCRAM stands for Salted Challenge Response Authentication Mechanism. In the diagram below, it illustrates a simple flow of SCRAM. The server would sends a random text (Star shape) over to the client, and the client would use this together with the password (Diamond shape) to compute…

Many years ago, setting up a Apache Kafka cluster on bare metal servers can be really complicated — storage sizing, determining the number of replication across brokers, number of topics and partitions, sourcing the better performance hardware for high IO operations and possibly high throughput within the internal network, designing…