AM RADIO: Reduce All Domains Into One Mike Ranzinger, Greg Heinrich, Jan Kautz, Pavlo Molchanov NVIDIA Research \[AM RADIO Paper\] \[PHI S Paper\] \[BibTex\]\[GitHub examples\] \[Tech report on v2.5\] HuggingFace Hub You can pull the model from a Python script: Usage RADIO will return a tuple with two tensors. The summary is similar to the cls token in ViT and is meant to represent the general concept of the entire image. It has shape $(B,C)$ with $B$ being the batch dimension, and $C$ being some number of channels. The spatial features represent more localized content which should be suitable for dense tasks such as semantic segmentation, or for integration into an LLM. It has shape $(B,T,D)$ with $T$ being the flattened spatial tokens, and $D$ being the channels for spatial features. Note that $C \neq D$ in general. Converting to a spatial tensor format can be done using the downsampling size of the model, combined with the input tensor shape. For 'radio v1', the patch size is 14. The resulting tensor will have shape $(B,D,H,W)$, as is typically seen with computer vision models. RADIOv2.5 Notes See the RADIOv2.5 technical report. License RADIO code and weights are released under…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy