Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Using EXO_OFFLINE to prevent runtime internet access
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors
- Verifying model weight integrity with SHA-256 checksums and signed model cards
- Running EXO in air-gapped networks without HuggingFace dependencies
Dashboard and API Access Control
- Installing and configuring reverse proxies (nginx, Caddy) with TLS termination
- Implementing role-based access control for the EXO dashboard and REST API
- Using macOS keychain or Linux pass to store secrets for API authentication
- Restricting administrative endpoints to specific source IP ranges
Cluster Isolation and Network Security
- Segmenting EXO clusters with EXO_LIBP2P_NAMESPACE and VLANs
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports
- Preventing unauthorized device discovery and rogue node injection
- Encrypting libp2p traffic between nodes when RDMA is not available
Model Governance and Provenance
- Building an internal model registry with approved model lists and metadata
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside source checkpoints
- Enforcing that only specific HuggingFace repos or internal artifacts can be loaded
- Documenting model lineage, license terms, and acceptable use policies
Audit Logging and Compliance
- Configuring EXO log forwarding to immutable audit trails (SIEM, WORM storage)
- Correlating API call logs with user identity and timestamp
- Capturing model instance creation, deletion, and inference request events
- Generating periodic compliance reports for internal and external auditors
Threat Modeling and Incident Response
- Identifying threats: data exfiltration through model outputs, prompt injection, side-channel leaks
- Implementing prompt monitoring and content filtering pipelines
- Creating incident response runbooks for cluster compromise scenarios
- Isolating affected nodes, preserving forensic logs, and rebuilding clean environments
Physical Security and Hardware Boundaries
- Securing Thunderbolt ports against unauthorized RDMA cable connections
- Using secure enclaves and Apple Silicon hardware attestation where applicable
- Controlling physical access to clustered Macs and shared storage
- Documenting hardware lifecycle and decommissioning procedures
Regulatory Considerations
- Mapping EXO deployments to GDPR, HIPAA, and SOC 2 requirements
- Maintaining data residency by keeping inference on-premise
- Documenting vendor supply-chain risks (MLX, EXO, model weights)
- Preparing for AI governance frameworks such as EU AI Act Article 53
Requirements
- Experience with EXO or another local LLM runtime
- Understanding of Unix filesystem permissions and networking ACLs
- Familiarity with TLS/SSL certificate management and encryption basics
Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators handling sensitive data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
