BiomedCLIP PubMedBERT 256 vit base patch16 224 BiomedCLIP is a biomedical vision language foundation model that is pretrained on PMC 15M, a dataset of 15 million figure caption pairs extracted from biomedical research articles in PubMed Central, using contrastive learning. It uses PubMedBERT as the text encoder and Vision Transformer as the image encoder, with domain specific adaptations. It can perform various vision language processing (VLP) tasks such as cross modal retrieval, image classification, and visual question answering. BiomedCLIP establishes new state of the art in a wide range of standard datasets, and substantially outperforms prior VLP approaches: Contents Training Data Model Use Reference Limitations Further Information Training Data We have released BiomedCLIP Data Pipeline at https://github.com/microsoft/BiomedCLIP data pipeline, which automatically downloads and processes a set of articles from the PubMed Central Open Access dataset. BiomedCLIP builds upon the PMC 15M dataset, which is a large scale parallel image text dataset generated by this data pipeline for biomedical vision language processing. It contains 15 million figure caption pairs extracted from bio…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy