flan t5 base for Extractive QA This is the flan t5 base model, fine tuned using the SQuAD2.0 dataset. It's been trained on question answer pairs, including unanswerable questions, for the task of Extractive Question Answering. UPDATE: With transformers version 4.31.0 the use remote code=True is no longer necessary. NOTE: The token must be manually added to the beginning of the question for this model to work properly. It uses the token to be able to make "no answer" predictions. The t5 tokenizer does not automatically add this special token which is why it is added manually. Overview Language model: flan t5 base Language: English Downstream task: Extractive QA Training data: SQuAD 2.0 Eval data: SQuAD 2.0 Infrastructure : 1x NVIDIA 3070 Model Usage Metrics Training procedure Training hyperparameters The following hyperparameters were used during training: learning rate: 2e 05 train batch size: 16 eval batch size: 8 seed: 42 gradient accumulation steps: 6 total train batch size: 96 optimizer: Adam with betas=(0.9,0.999) and epsilon=1e 08 lr scheduler type: linear lr scheduler warmup ratio: 0.1 num epochs: 4.0 Training results Framework versions Transformers 4.30.0.dev0 Pytorch 2.0.1…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy