View the project page: https://meta math.github.io/ see our paper at https://arxiv.org/abs/2309.12284 Note All MetaMathQA data are augmented from the training sets of GSM8K and MATH. None of the augmented data is from the testing set. You can check the original question in meta math/MetaMathQA , each item is from the GSM8K or MATH train set. Model Details MetaMath Mistral 7B is fully fine tuned on the MetaMathQA datasets and based on the powerful Mistral 7B model. It is glad to see using MetaMathQA datasets and changing the base model from llama 2 7B to Mistral 7b can boost the GSM8K performance from 66.5 to 77.7 . To fine tune Mistral 7B, I would suggest using a smaller learning rate (usually 1/5 to 1/10 of the lr for LlaMa 2 7B) and staying other training args unchanged. More training details and scripts can be seen at https://github.com/meta math/MetaMath. Installation Model Usage prompting template: ''' "Below is an instruction that describes a task. " "Write a response that appropriately completes the request.\n\n" " Instruction:\n{instruction}\n\n Response: Let's think step by step." ''' where you need to use your query question to replace the {instruction} There is another i…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy