Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Multimodal LLMs in Vertex AI
- Overview of multimodal features available in Vertex AI
- Gemini models and their supported modalities
- Enterprise and research use cases
Setting Up the Development Environment
- Configuring Vertex AI for multimodal workflows
- Managing datasets across various modalities
- Practical lab: environment setup and dataset preparation
Long Context Windows and Advanced Reasoning
- Concepts behind long-context workflows
- Applications in planning and decision-making processes
- Practical lab: implementing long-context analysis
Cross-Modal Workflow Design
- Integrating text, audio, and image analysis
- Chaining multimodal steps within pipelines
- Practical lab: designing a multimodal pipeline
Working with Gemini API Parameters
- Configuring multimodal inputs and outputs
- Optimising inference speed and efficiency
- Practical lab: tuning Gemini API parameters
Advanced Applications and Integrations
- Building interactive multimodal agents and assistants
- Integrating external APIs and tools
- Practical lab: developing a multimodal application
Evaluation and Iteration
- Testing multimodal performance
- Metrics for accuracy, alignment, and drift detection
- Practical lab: evaluating multimodal workflows
Summary and Next Steps
Requirements
- Strong proficiency in Python programming
- Experience in developing machine learning models
- Familiarity with multimodal data types (text, audio, image)
Target Audience
- AI researchers
- Advanced developers
- ML scientists
