electra small discriminator CoLA This model is a fine tuned version of google/electra small discriminator on the GLUE COLA dataset. It achieves the following results on the evaluation set: Loss: 0.4403 Matthews Correlation: 0.5510 Model description trying to optimize accuracy/speed: Intended uses & limitations More information needed Training and evaluation data More information needed Training procedure Training hyperparameters The following hyperparameters were used during training: learning rate: 8e 05 train batch size: 512 eval batch size: 16 seed: 32754 distributed type: multi GPU optimizer: Adam with betas=(0.9,0.999) and epsilon=1e 08 lr scheduler type: cosine lr scheduler warmup ratio: 0.05 num epochs: 8.0 mixed precision training: Native AMP Training results Training Loss Epoch Step Validation Loss Matthews Correlation : : : : : : : : : : 0.6139 1.0 17 0.5997 0.0 0.5315 2.0 34 0.4890 0.5154 0.4244 3.0 51 0.4469 0.5433 0.3568 4.0 68 0.4403 0.5510 0.319 5.0 85 0.4517 0.5654 0.2887 6.0 102 0.4656 0.5728 0.2771 7.0 119 0.4558 0.5883 0.2729 8.0 136 0.4569 0.5858 Framework versions Transformers 4.27.0.dev0 Pytorch 1.13.1+cu117 Datasets 2.8.0 Tokenizers 0.13.1
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy