Splinter base model Splinter base is the pretrained model discussed in the paper Few Shot Question Answering by Pretraining Span Selection (at ACL 2021). Its original repository can be found here. The model is case sensitive. Note: This model doesn't contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model with those weights, see tau/splinter base qass. Model description Splinter is a model that is pretrained in a self supervised fashion for few shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was pretrained with the Recurring Span Selection (RSS) objective, which emulates the span selection process involved in extractive question answering. Given a text, clusters of recurring spans (n grams that appear more than once in the text) are first identified. For each such cluster, all of its instances but one are replaced with a special [QUESTION] token, and the model sho…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy