Building Custom AI Operators with CANN TIK and TVM Training Course

Introduction to Custom Operator Development

• The rationale behind building custom operators: use cases and constraints
• The architecture of the CANN runtime and key operator integration points
• An overview of TBE, TIK, and TVM within the Huawei AI ecosystem

Utilising TIK for Low-Level Operator Programming

• Comprehending the TIK programming model and its supported APIs
• Memory management strategies and tiling approaches in TIK
• Creating, compiling, and registering a custom operator with CANN

Testing and Validating Custom Operators

• Conducting unit and integration testing of operators within the graph
• Diagnosing and resolving kernel-level performance issues
• Visualising operator execution flows and buffer behaviours

TVM-Based Scheduling and Optimisation

• Understanding TVM as a compiler for tensor operations
• Crafting schedules for custom operators in TVM
• TVM tuning, benchmarking, and code generation specifically for Ascend

Integration with Frameworks and Models

• Registering custom operators for compatibility with MindSpore and ONNX
• Verifying model integrity and managing fallback behaviour
• Supporting multi-operator graphs with mixed precision capabilities

Case Studies and Specialised Optimisations

• Case study: Achieving high-efficiency convolution for small input shapes
• Case study: Optimising attention operators with memory constraints in mind
• Best practices for deploying custom operators across various devices

Summary and Next Steps