Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
NVIDIA AI and GPU Infrastructure Foundations
Objective: Establish a shared technical baseline and align AI workload characteristics with appropriate GPU infrastructure selections.
AI Workloads and Accelerated Computing
- Concepts surrounding AI, machine learning, deep learning, generative AI, and modern AI factories.
- Distinguishing between training, fine-tuning, and inference, including their respective resource and performance profiles.
- Understanding the behavioural differences between CPUs and GPUs when handling parallel AI workloads.
NVIDIA GPU Architecture
- Exploration of GPU generations, streaming multiprocessors, CUDA cores, and Tensor Cores.
- Considerations regarding device memory, bandwidth, caches, and data movement.
- The role of PCIe, SXM, NVLink, and NVSwitch within single- and multi-GPU systems.
- Utilizing NVIDIA utilities to interpret device identity, topology, and capabilities.
NVIDIA Hardware and Software Ecosystem
- The function of CPUs, GPUs, and BlueField DPUs in accelerated infrastructure setups.
- Overview of DGX, HGX, and NVIDIA-Certified server concepts, along with their deployment models.
- Components such as drivers, CUDA, container runtimes, NGC, and NVIDIA AI Enterprise.
- Evaluating deployment options across bare metal, virtualized, Kubernetes, and cloud environments.
AI Infrastructure Building Blocks
- Responsibilities associated with compute, network, storage, and management planes.
- Using reference architectures and support matrices as tools for design and validation.
- Foundational concepts in monitoring, orchestration, MLOps, and job scheduling.
HANDS-ON LAB | Discover and Validate a GPU Node
- Examining GPU model, driver, firmware-visible state, and CUDA compatibility.
- Analyzing GPU-to-CPU and GPU-to-GPU topology to pinpoint potential bottlenecks.
- Establishing a baseline inventory to be utilized in subsequent virtualization and monitoring exercises.
AI Server Architecture and Infrastructure Deployment
Objective: Convert workload requirements into a sustainable AI server, network, and storage design.
AI Server Platforms and Topology
- Characteristics of PCIe and SXM platforms, alongside single- and multi-GPU design patterns.
- Factors related to CPU architecture, memory, NUMA locality, and PCIe lane configuration.
- NVLink/NVSwitch topology, GPU peer access, and scale-up communication strategies.
- Prerequisites for BMC, firmware, BIOS, and operating systems on GPU servers.
Networking for AI Workloads
- The roles of Ethernet and InfiniBand in training and inference environments.
- Considerations for RDMA, RoCE, GPUDirect RDMA, and latency/bandwidth constraints.
- Separation of north-south, east-west, and management networks.
- Strategies for fabric redundancy, observability, and addressing common operational failures.
Storage and the AI Data Path
- Access patterns for datasets, checkpoints, models, and inference serving.
- Evaluations of local, shared file, object, and parallel storage solutions.
- Concepts involving throughput, IOPS, metadata performance, and GPUDirect Storage.
Deployment and Operational Readiness
- Sizing compute, memory, network, and storage resources based on workload demands.
- Validation of power, cooling, rack, cabling, and environmental factors.
- Alignment of support matrices, firmware/drivers, and acceptance testing procedures.
- Planning for capacity headroom, resilience, and lifecycle management.
HANDS-ON LAB | Design and Validate an AI Infrastructure Blueprint
- Mapping workload profiles to specific server, GPU, network, and storage requirements.
- Reviewing topology to identify risks related to oversubscription, NUMA, or data paths.
- Constructing a concise deployment-readiness and acceptance checklist.
GPU Virtualization, NVIDIA vGPU and VFIO Passthrough
Objective: Select and implement the most suitable GPU assignment model for virtualized AI workloads.
GPU Resource Assignment Models
- Comparison of full physical GPU assignment, time-sliced vGPU, MIG-backed vGPU, and MIG.
- Trade-offs involving isolation, predictability, density, live operations, and licensing.
- Placement patterns for development, inference, fine-tuning, and shared services workloads.
NVIDIA vGPU Architecture and Administration
- Components such as the vGPU Manager, guest driver, profile selection, and frame-buffer allocation.
- Validation of supported hypervisors, guest operating systems, and compatibility.
- Management of NVIDIA license services, entitlements, and operational dependencies.
- Processes for provisioning, validation, capacity control, and addressing common failure modes.
Linux VFIO GPU Passthrough
- Concepts regarding IOMMU, PCIe device groups, and platform prerequisites.
- Identifying GPU and companion devices, and binding them to vfio-pci.
- Assigning a GPU to a virtual machine and installing the necessary guest driver.
- Verifying device isolation and resolving driver-binding conflicts.
Performance and Troubleshooting
- Considerations for NUMA placement, CPU pinning, huge pages, and memory locking.
- Diagnostics for host/guest driver compatibility and device visibility.
- Criteria for deciding between passthrough and vGPU in enterprise operations.
HANDS-ON LAB | Assign and Validate GPU Resources in a VM
- Verifying IOMMU and device grouping, then preparing the GPU for passthrough or vGPU assignment.
- Attaching the GPU resource, installing or validating guest components, and confirming workload access.
- Diagnosing pre-configured device-visibility or compatibility faults.
Multi-Instance GPU and NVIDIA AI Enterprise
Objective: Partition supported GPUs using MIG and deploy managed AI software components via NVIDIA AI Enterprise.
MIG Architecture and Use Cases
- Spatial GPU partitioning, hardware isolation, and predictable resource allocation.
- Overview of GPU instances, compute instances, profiles, and supported configuration patterns.
- Comparing MIG with time slicing, vGPU, and full-GPU assignment.
- Appropriate use cases for inference, development, smaller training tasks, and multi-tenancy.
MIG Configuration and Lifecycle
- Enabling MIG mode and managing the creation, listing, and removal of instances.
- Mapping profiles to workloads and exposing devices to containers or schedulers.
- Understanding MIG-backed vGPU concepts and platform-specific limitations.
- Considerations for reconfiguration, maintenance, persistence, and troubleshooting.
NVIDIA AI Enterprise
- Overview of platform architecture, the infrastructure layer, and the application software layer.
- Aspects of licensing, NGC access, release branches, and lifecycle/support considerations.
- Components including drivers, NVIDIA Container Toolkit, GPU Operator, and supported deployment patterns.
- Deployment paths for bare metal, virtualized, and Kubernetes environments.
Containerized AI Workloads
- Accessing approved containers and artifacts from NGC.
- Executing GPU-enabled containers and validating resource visibility.
- Managing security, image provenance, secrets, and controlled operational access.
HANDS-ON LAB | Partition a GPU and Deploy a Workload
- Creating a supported MIG configuration and verifying instance allocation.
- Exposing a selected GPU or MIG device to a containerized workload.
- Validating isolation, utilization, and basic workload health.
Monitoring, Operations, Troubleshooting and Capstone
Objective: Manage NVIDIA AI infrastructure using consistent monitoring, incident, change, and capacity-management practices.
GPU Monitoring and Observability
- Using nvidia-smi and NVIDIA DCGM to inspect health and utilization.
- Core metrics including compute, memory, power, temperature, clocks, PCIe/NVLink, and ECC.
- Integration patterns with DCGM Exporter, Prometheus, and Grafana.
- Setting baselines, alert thresholds, dashboards, and tracking capacity trends.
Troubleshooting and Performance
- Addressing GPU discovery failures, driver/library mismatches, and container-runtime issues.
- Investigating Xid events, ECC errors, thermal or power throttling, and resource contention.
- Distinguishing issues arising from infrastructure, virtualization, schedulers, or workloads.
- Collecting evidence, performing safe remediation, escalating, and conducting post-incident reviews.
Operational Best Practices
- Planning the lifecycle of drivers, firmware, and NVIDIA AI Enterprise components.
- Managing change windows, rollback readiness, and compatibility validation.
- Handling capacity management, workload placement, and service-level considerations.
- Maintaining access control, auditability, image governance, and operational documentation.
- Creating runbooks for health checks, incident response, and routine maintenance.
HANDS-ON LAB | Final Operational Capstone
- Reviewing a scenario and selecting an appropriate GPU resource-assignment model.
- Validating the environment, deploying or inspecting a workload, and establishing monitoring.
- Diagnosing injected symptoms and documenting the evidence-based remediation path.
- Presenting a concise deployment and operations runbook for peer review.
Assessment and Completion
- Brief knowledge checks at the conclusion of each day.
- Completion of guided practical exercises and validation tasks.
- Participation in the final troubleshooting and operations capstone.
- Issuance of a NobleProg course completion certificate to participants meeting the agreed criteria.
Recommended Lab Environment
The final configuration is contingent upon the client's target platform. A suitable lab typically includes:
- An NVIDIA data-center GPU platform supporting the selected vGPU and/or MIG exercises.
- A supported hypervisor or Linux/KVM host with IOMMU/VFIO enabled where applicable.
- Supported Linux guest or bare-metal operating systems with administrative access.
- Required NVIDIA drivers, vGPU components, and license service access.
- NVIDIA AI Enterprise entitlement or evaluation access, along with an NGC account/API key.
- Docker or another supported container runtime; Kubernetes where agreed for the delivery.
- Monitoring components such as DCGM, DCGM Exporter, Prometheus, and Grafana.
- Internet or registry access, or pre-staged packages and container images for restricted sites.
LAB ASSUMPTION | Hands-on vGPU, VFIO, and MIG activities require compatible hardware, software, and entitlements. NobleProg will validate the lab bill of materials and adapt exercises before delivery. Where a live configuration is not feasible, the affected exercise may use an instructor demonstration or prepared simulation.
Participant Deliverables
- NobleProg course materials and lab instructions.
- GPU infrastructure validation checklist.
- Sample monitoring and operational runbook structure.
- NobleProg electronic certificate of attendance or completion.
Requirements
Participants should bring a practical infrastructure background and access to the agreed laboratory environment.
- Working knowledge of Linux and command-line operations
- Familiarity with server hardware, networking, and storage
- Basic understanding of virtual machines and hypervisors
- Foundational knowledge of AI, machine learning, and GPU workloads
- Container and Kubernetes awareness is helpful, not mandatory
- Administrative lab access for practical exercises
70 Hours
