Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
- Introduction
- Hadoop history and foundational concepts
- The Hadoop ecosystem
- Available distributions
- High-level architecture overview
- Dispelling Hadoop myths
- Challenges related to hardware and software
- Labs: Discussing your Big Data projects and challenges
- Planning and installation
- Selecting software and Hadoop distributions
- Cluster sizing and growth planning
- Choosing hardware and network infrastructure
- Rack topology design
- Installation procedures
- Multi-tenancy considerations
- Directory structure and logging
- Benchmarking processes
- Labs: Cluster installation and performance benchmarking
- HDFS operations
- Core concepts (horizontal scaling, replication, data locality, rack awareness)
- Nodes and daemons (NameNode, Secondary NameNode, HA Standby NameNode, DataNode)
- Health monitoring
- Administration via command-line and browser interfaces
- Adding storage and replacing defective drives
- Labs: Familiarization with HDFS command-line tools
- Data ingestion
- Using Flume for log and other data ingestion into HDFS
- Importing from SQL databases to HDFS and exporting back using Sqoop
- Hadoop data warehousing with Hive
- Copying data between clusters (distcp)
- Utilizing S3 as a complement to HDFS
- Data ingestion best practices and architectures
- Labs: Setting up and utilizing Flume and Sqoop
- MapReduce operations and administration
- Parallel computing precedents: comparing HPC vs Hadoop administration
- MapReduce cluster loads
- Nodes and Daemons (JobTracker, TaskTracker)
- Walkthrough of the MapReduce UI
- MapReduce configuration
- Job configuration
- Optimizing MapReduce
- Error prevention in MR: Guidance for programmers
- Labs: Executing MapReduce examples
- YARN: new architecture and capabilities
- YARN design goals and implementation architecture
- New components: ResourceManager, NodeManager, Application Master
- Installing YARN
- Job scheduling under YARN
- Labs: Investigating job scheduling
- Advanced topics
- Hardware monitoring
- Cluster monitoring
- Adding/removing servers and upgrading Hadoop
- Backup, recovery, and business continuity planning
- Oozie job workflows
- Hadoop high availability (HA)
- Hadoop Federation
- Securing your cluster with Kerberos
- Labs: Configuring monitoring systems
- Optional tracks
- Cloudera Manager for cluster administration, monitoring, and routine tasks, including installation and usage. In this track, all exercises and labs are conducted within the Cloudera distribution environment (CDH5)
- Ambari for cluster administration, monitoring, and routine tasks, including installation and usage. In this track, all exercises and labs are conducted within the Ambari cluster manager and Hortonworks Data Platform (HDP 2.0)
Requirements
- Proficiency in basic Linux system administration
- Fundamental scripting skills
Prior knowledge of Hadoop and Distributed Computing is not necessary, as these concepts will be introduced and explained during the course.
Lab environment
Zero Installation: There is no requirement to install Hadoop software on students’ machines! A functional Hadoop cluster will be provided for use.
Students will need the following:
- An SSH client (Linux and Mac typically include SSH clients; Putty is recommended for Windows users)
- A browser to access the cluster. We suggest using the Firefox browser with the FoxyProxy extension installed
Testimonials (1)
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
