Multi-modal
Multi-modal AI refers to artificial intelligence systems that can process and understand information from multiple types of data, such as text, images, audio, and video, simultaneously.
How it works
Instead of being limited to a single format like text, a multi-modal model combines different 'modes' of input to build a more complete understanding, similar to how humans use sight and sound together. This allows AI to perform more complex and nuanced tasks, like describing a picture or answering questions about a video.
Examples
- 1.An AI that can analyze a photo of a broken appliance and then generate step-by-step text instructions for repair.
- 2.A virtual assistant that responds to a user's spoken question by showing a relevant chart and summarizing it in text.
- 3.A content moderation system that reviews a social media post by checking its image, caption text, and any background audio together.