Reformer Language model on character level and trained on enwik8. enwik8 is a dataset based on Wikipedia and is often used to measure the model's ability to compress data, e.g. in the scope of the Hutter prize : https://en.wikipedia.org/wiki/Hutter Prize. reformer enwik8 was pretrained on the first 90M chars of enwik8 whereas the text was chunked into batches of size 65536 chars (=2^16). The model's weights were taken from https://console.cloud.google.com/storage/browser/trax ml/reformer/enwik8 and converted to Hugging Face's PyTorch ReformerLM model ReformerModelWithLMHead . The model is a language model that operates on characters. Therefore, this model does not need a tokenizer. The following function can instead be used for encoding and decoding : Text can be generated as follows: Note : Language generation using ReformerModelWithLMHead is not optimized yet and is rather slow.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy