Building Multimodal Search and RAG - Syllabus

• Introduction
  • Introduction to multimodal search and RAG applications
  • Course objectives and key concepts
• Overview of Multimodality
  • Implementing contrastive learning for multimodal models
  • Building modality-independent embeddings
• Multimodal Search
  • Building a search system that retrieves across different modalities
  • Any-to-any retrieval techniques
• Large Multimodal Models (LMMs)
  • Understanding LMMs and visual instruction tuning
  • Practical applications for image reasoning
• Multimodal RAG (MM-RAG)
  • Building a multimodal RAG system
  • Generating responses based on multimodal context
• Industry Applications
  • Real-world applications in document analysis
  • Extracting structured data from images, invoices, and flowcharts
• Multimodal Recommender System
  • Implementing a multi-vector recommender system
  • Using similarity comparison across multiple modalities
• Conclusion
  • Summary of course topics and next steps for multimodal AI
• Appendix - Tips and Help
  • Additional resources and troubleshooting tips
  • Code examples for common challenges