Audio Classification and Event Detection with ML Training Course

Foundations of Audio Classification

• Categorising sound events: environmental, mechanical, and human-generated.
• Exploring use cases such as surveillance, monitoring, and automation.
• Distinguishing between audio classification, detection, and segmentation.

Audio Data and Feature Extraction

• Recognising various audio file types and formats.
• Considering sampling rates, windowing, and frame sizes.
• Extracting features like MFCCs, chroma, and mel-spectrograms.

Data Preparation and Annotation

• Working with datasets including UrbanSound8K, ESC-50, and custom collections.
• Labelling sound events and their temporal boundaries.
• Balancing datasets and applying audio augmentation techniques.

Building Audio Classification Models

• Applying convolutional neural networks (CNNs) to audio data.
• Evaluating model inputs: raw waveforms versus extracted features.
• Utilising appropriate loss functions, evaluation metrics, and strategies to prevent overfitting.

Event Detection and Temporal Localisation

• Implementing frame-based and segment-based detection strategies.
• Post-processing detections using thresholds and smoothing techniques.
• Visualising predictions along audio timelines.

Advanced Topics and Real-Time Processing

• Employing transfer learning in scenarios with limited data.
• Deploying models using TensorFlow Lite or ONNX.
• Addressing streaming audio processing and latency considerations.

Project Development and Application Scenarios

• Designing a comprehensive pipeline from data ingestion to classification.
• Creating a proof-of-concept for surveillance, quality control, or monitoring.
• Integrating logging, alerting, and connections to dashboards or APIs.

Summary and Next Steps