Administration of Confluent Apache Kafka Training Course

Module 1: Introduction to the architecture and configuration of the Confluent Apache Kafka cluster

• The role of Kafka within modern data pipelines
• Distinguishing between Apache Kafka and Confluent Kafka
• Core components: producers, consumers, brokers, topics, and partitions
• Deployment models and scaling considerations for Kafka clusters

Module 2: Zookeeper Quorum Configuration

• An introduction to Zookeeper
• The function of Zookeeper within a Kafka cluster
• Determining the size of the Zookeeper Quorum
• Configuring Zookeeper
• Setting up SSH on our servers
• Hands-on: Configuring Zookeeper as a team and as a service
• Utilising the Zookeeper Command Line Interface (CLI)
• Hands-on: Configuring the Zookeeper Quorum
• The Zookeeper internal file system
• Performance factors impacting Zookeeper
• Demonstration of management tools for Zookeeper and Zoonavigator

Module 3: Kafka Cluster Configuration

• Fundamental Kafka concepts
• General Kafka configuration
• Hands-on: Configuring the Kafka broker
• Hands-on: Running Kafka commands
• Hands-on: Configuring a Kafka Multi-Broker Cluster
• Hands-on: Testing the Kafka cluster
• Verifying connectivity to the Kafka cluster
• The Advertised.listeners setting: the most critical configuration
• Topic configuration details
• Settings for downloading and ingesting messages into topics
• Hands-on: Demonstrating Kafka resilience
• Kafka performance: I/O operations
• Kafka performance: Network (RED)
• Kafka performance: RAM usage
• Kafka performance: CPU utilisation
• Kafka performance: Operating System (OS) impact
• Kafka performance: Other considerations
• Hands-on: Modifying Kafka broker configuration

Module 4: Advanced Kafka Configuration

• Configuring Landoop Kafka topic UI, Confluent REST Proxy, and Confluent Schema Registry
• Transmitting and receiving messages (via CLI, Java, and Spring framework)
• Monitoring metrics and tools (including Confluent Control Center and Elasticsearch)
• Log files and offset management
• High availability and disaster recovery planning
• Maintaining high availability through replication
• Optimising producer and consumer performance
• Disaster recovery strategies
• Failover control and data recovery procedures
• Connector configuration
• Implementing Kafka Connect
• Securing Kafka environments

Summary and Next Steps