Model Card for BabyLM submission to the Interaction Track A 124M model with the GPT 2 architecture trained with the next token prediction loss for 10 epochs (~900 M words) on 90% of the BabyLM corpus and an additional 1 M words of PPO RL training as submission for the Interaction track of the 2025 BabyLM challenge. This model card is based on the model card of the BabyLM 100M GPT 2 baseline. Table of Contents Model Card for Storytelling Submission Model Table of Contents Model Details Model Description Uses Training Details Training Data Hyperparameters Training Procedure Size and Checkpoints Evaluation Testing Data & Metrics Testing Data Metrics Results Technical Specifications Model Architecture and Objective Compute Infrastructure Hardware Software Training Time Citation Model Card Authors Bibliography Model Details Model Description This is the RL storytelling model, based on a pretrained GPT 2 model, for the Interaction Track of the 2025 BabyLM challenge. Developed by: Jonas Mayer Martins, Ali Hamza Bashir, Muhammad Rehan Khalid Model type: Causal language model Language(s) (NLP): eng Resources for more information: GitHub Repo Uses This is a pre trained language model. It can…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy