Multilingual clip: XLM Roberta Large Vit B 16Plus Multilingual CLIP extends OpenAI's English text encoders to multiple other languages. This model only contains the multilingual text encoder. The corresponding image model Vit B 16Plus can be retrieved via instructions found on mlfoundations open clip repository on Github. We provide a usage example below. Requirements To use both the multilingual text encoder and corresponding image encoder, we need to install the packages multilingual clip and open clip torch . Usage Extracting embeddings from the text encoder can be done in the following way: Extracting embeddings from the corresponding image encoder: Evaluation results None of the M CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS COCO dataset, we see the following R@10 results: Name En De Es Fr Zh It Pl Ko Ru Tr Jp : : : : : : : : : : : : : : : : : : : : : : OpenAI CLIP Vit B/32 90.3 OpenAI CLIP Vit L/14 91.8 OpenCLIP ViT B 16+ 94.3 LABSE Vit L/14 91.6 89.6 89.5 89.9 88.9 90.1 89.8 80.8 85.5 89.8 73.9 XLM R Large Vit B/32 91.8 88.7 89.1 89.4 89.3 89.8 91.4 82.1 86.1 88.8 81.0 XLM R Vit L/14 92.4 90.6 91.0 90.0 89.7 9…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy