1. Embeddings represent real-world objects as arrays of numbers for machine learning models
2. Embeddings are used in various ML applications such as semantic search, recommendations, and named entity extraction
3. BigQuery now supports multimodal embeddings and embedding generation for structured data using PCA or autoencoder models
Embeddings are numerical representations of real-world objects like entities, text, images, or videos that can be easily processed by machine learning models. They are essential for various ML applications such as semantic search, recommendations, and outlier detection. Last year, text embeddings were introduced in BigQuery, allowing for more effective understanding of real-world data domains. This year, vector search was also introduced for working with billions of embeddings and building generative AI applications on BigQuery.
At Next ’24, enhancements to embedding generation capabilities in BigQuery were announced. This includes support for multimodal embeddings through Vertex AI, allowing for the embedding of text and image data in the same semantic space. Additionally, embedding generation for structured data using models like PCA, Autoencoder, or Matrix Factorization can now be trained on data in BigQuery.
Multimodal embeddings generate vectors for both text and image data in a shared semantic space, enabling various use cases such as indexing images and searching for them via text. To start using multimodal embeddings in BigQuery, you can create an object table pointing to your unstructured data, such as images stored in a Google Cloud Storage bucket. It is important to ensure that the necessary permissions and API access are set up for using Vertex AI in BigQuery to generate embeddings effectively.