sentence transformers/clip ViT B 32 multilingual v1 This is a multi lingual version of the OpenAI CLIP ViT B32 model. You can map text (in 50+ languages) and images to a common dense vector space such that images and the matching texts are close. This model can be used for image search (users search through a large collection of images) and for multi lingual zero shot image classification (image labels are defined as text). Usage (Sentence Transformers) Using this model becomes easy when you have sentence transformers installed: Then you can use the model like this: Multilingual Image Search Demo For a demo of multilingual image search, have a look at: Image Search multilingual.ipynb ( Colab version ) For more details on image search and zero shot image classification, have a look at the documentation on SBERT.net. Training This model has been created using Multilingual Knowledge Distillation. As teacher model, we used the original clip ViT B 32 and then trained a multilingual DistilBERT model as student model. Using parallel data, the multilingual student model learns to align the teachers vector space across many languages. As a result, you get an text embedding model that works…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy