Multilingual clip: XLM Roberta Large Vit L 14 Multilingual CLIP extends OpenAI's English text encoders to multiple other languages. This model only contains the multilingual text encoder. The corresponding image model ViT L 14 can be retrieved via instructions found on OpenAI's CLIP repository on Github. We provide a usage example below. Requirements To use both the multilingual text encoder and corresponding image encoder, we need to install the packages multilingual clip and clip . Usage Extracting embeddings from the text encoder can be done in the following way: Extracting embeddings from the corresponding image encoder: Evaluation results None of the M CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS COCO dataset, we see the following R@10 results: Name En De Es Fr Zh It Pl Ko Ru Tr Jp : : : : : : : : : : : : : : : : : : : : : : OpenAI CLIP Vit B/32 90.3 OpenAI CLIP Vit L/14 91.8 OpenCLIP ViT B 16+ 94.3 LABSE Vit L/14 91.6 89.6 89.5 89.9 88.9 90.1 89.8 80.8 85.5 89.8 73.9 XLM R Large Vit B/32 91.8 88.7 89.1 89.4 89.3 89.8 91.4 82.1 86.1 88.8 81.0 XLM R Vit L/14 92.4 90.6 91.0 90.0 89.7 91.1 91.3 85.2 85.8 90.3 81.9…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy