10th October 2023 (9 Topics)
2
Oct
3
Oct
4
Oct
5
Oct
6
Oct
7
Oct
9
Oct
10
Oct
11
Oct
12
Oct
13
Oct
14
Oct
16
Oct
17
Oct
18
Oct
19
Oct
20
Oct
21
Oct
23
Oct
24
Oct
25
Oct
26
Oct
27
Oct
28
Oct
30
Oct
31
Oct
Multimodal artificial intelligence
Context
After a report by “The Information” revealed that Google’s new yet-to-be-released multimodal large language model called ‘Gemini’ was already being tested in a bunch of companies.
|
Google is also working on a new project called‘Gobi’ which is expected to be a multimodal AI system from scratch, unlike the GPT models. |
About
About multimodal AI:
- Multimodal AI combines different types of information like text, images, and audio to perform various tasks, such as detecting hateful memes or predicting dialogue lines in videos.
- Models like OpenAI's DALL.E use this approach to generate images based on text prompts, by finding patterns that connect visual data with image descriptions.
- In the case of audio, OpenAI's Whisper, a speech-to-text translation model, enables the system to recognize speech in audio and convert it into simple text.
Applications of Multimodal AI:
- Meta introduced a complex open-source AI system called ImageBind, which incorporates text, visual data, audio, temperature, and movement readings.
- This system hints at the possibility of future AI including more sensory data like touch, smell, and brain signals.
- Industries like medicine and autonomous driving benefit from multimodal AI.
- It helps analyze complex datasets in areas like identifying rare genetic variations and processing CT scans.
- Additionally, speech translation models like Google Translate use multiple modes for efficient translation across different languages.
Courses
Study
Notes
Home
Resources
More
