Friday
Room 6 - Level 5
15:00 - 16:00
(UTC±00)
Talk (60 min)
Using Vector Databases for Multimodal Embeddings and Search
Many real-world problems are inherently multimodal, from the communicative modalities humans use such as spoken language and gestures to the force, proprioception, and visual sensors ubiquitous in robotics. In order for machine learning models to address these problems and interact more naturally and wholistically with the world around them and ultimately be more general and powerful reasoning engines we need them to understand data across all of its corresponding image, video, text, audio, and tactile representations.
In this talk, Zain Hasan will discuss how we can use open-source multimodal models (such as https://github.com/facebookresearch/ImageBind), that can see, hear, read, and feel data(!), to perform cross-modal search(searching audio with images, videos with text etc.) at the billion-object scale with the help of open source vector databases. I will also demonstrate, with live code demos and large-scale datasets, how being able to perform this cross-modal retrieval in real-time can help users add natural search interfaces to their apps. This talk will revolve around how we scaled the usage of multimodal embedding models in production and how you can add cross-modal search into your apps.
Zain Hasan
Zain Hasan is a Senior Developer Advocate at Weaviate an open-source vector database. He is an engineer and data scientist by training, who pursued his undergraduate and graduate work at the University of Toronto building artificially intelligent assistive technologies. He then founded his company developing a digital health platform that leveraged machine learning to remotely monitor chronically ill patients. More recently he practiced as a consultant senior data scientist in Toronto. He is passionate about open-source software, education, community, and machine learning and has delivered workshops and talks at multiple events and conferences.