Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure.

• Biological versus artificial neurons.
• The mathematical model of an ANN.
• Activation functions employed in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning Mechanisms.

• Review of vector and matrix algebra.
• State-space concepts.
• Principles of optimization.
• Error-correction learning.
• Memory-based learning.
• Hebbian learning.
• Competitive learning.

Single-Layer Perceptrons.

• Perceptron structure and learning processes.
• Pattern classification - introduction and Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• Perceptron convergence.
• Limitations associated with perceptrons.

Feedforward ANNs.

• Architecture of multi-layer feedforward networks.
• The backpropagation algorithm.
• Backpropagation - training and convergence dynamics.
• Functional approximation via backpropagation.
• Practical and design considerations in backpropagation learning.

Radial Basis Function Networks.

• Pattern separability and interpolation.
• Theory of regularization.
• Regularization in the context of RBF networks.
• Design and training of RBF networks.
• Approximation characteristics of RBFs.

Competitive Learning and Self-Organizing ANNs.

• General clustering methodologies.
• Learning Vector Quantization (LVQ).
• Competitive learning algorithms and their architectures.
• Self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks.

• Neuro-fuzzy systems.
• Foundations of fuzzy sets and logic.
• Design of fuzzy systems.
• Construction of fuzzy ANNs.

Applications

• A discussion on specific examples of Neural Network applications, highlighting their benefits and associated challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets - consistent case
  • Guarantees for finite hypothesis sets - inconsistent case
  • General considerations
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection
• Rademacher Complexity and VC Dimension
• The Bias-Variance trade-off
• Regularization
• Overfitting
• Validation
• Support Vector Machines
• Kriging (Gaussian Process Regression)
• PCA and Kernel PCA
• Self-Organizing Maps (SOM)
• Kernel-induced vector spaces
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This module is taught in conjunction with the topics covered on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications