MiniCheck Flan T5 Large This is a fact checking model from our work: 📃 MiniCheck: Efficient Fact Checking of LLMs on Grounding Documents ,(EMNLP 2024, GitHub Repo) The model is based on Flan T5 Large that predicts a binary label 1 for supported and 0 for unsupported. The model is doing predictions on the sentence level . It takes as input a document and a sentence and determine whether the sentence is supported by the document: MiniCheck Model(document, claim) {0, 1} MiniCheck Flan T5 Large is the best fack checking model with size The performance of these models is evaluated on our new collected benchmark (unseen by our models during training), LLM AggreFact, from 11 recent human annotated datasets on fact checking and grounding LLM generations. MiniCheck Flan T5 Large outperform all exisiting specialized fact checkers with a similar scale by a large margin (4 10% absolute increase) and is on par with GPT 4, but 400x cheaper. See full results in our work. Note: We only evaluated the performance of our models on real claims without any human intervention in any format, such as injecting certain error types into model generated claims. Those edited claims do not reflect LLMs' actua…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy