MiniCheck DeBERTa v3 Large This is a fact checking model from our work: 📃 MiniCheck: Efficient Fact Checking of LLMs on Grounding Documents (EMNLP 2024, GitHub Repo) The model is based on DeBERTa v3 Large that predicts a binary label 1 for supported and 0 for unsupported. The model is doing predictions on the sentence level . It takes as input a document and a sentence and determine whether the sentence is supported by the document: MiniCheck Model(document, claim) {0, 1} MiniCheck DeBERTa v3 Large is fine tuned from microsoft/deberta v3 large (He et al., 2023) on the combination of 35K data: 21K ANLI data (Nie et al., 2020) 14K synthetic data generated from scratch in a structed way (more details in the paper). Model Variants bespokelabs/Bespoke Minicheck 7B (Model Size: 7B) lytang/MiniCheck Flan T5 Large (Model Size: 0.8B) lytang/MiniCheck RoBERTa Large (Model Size: 0.4B) Model Performance The performance of these models is evaluated on our new collected benchmark (unseen by our models during training), LLM AggreFact, from 11 recent human annotated datasets on fact checking and grounding LLM generations. MiniCheck DeBERTa v3 Large outperform all exisiting specialized fact checke…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy