Importance Matrix Calibration Datasets This repository provides calibration datasets used to generate importance matrices (imatrix), which are required to minimize errors when quantizing models with LLaMA C++. The llama imatrix program cannot handle parquet files directly and thus requires them to be converted into text format first. There are many ways to do this but a simple approach is to use DuckDB with the following command: duckdb noheader ascii c "SELECT content FROM 'tools micro.parquet';" tools micro.txt Code calibration datasets This dataset consists of cleaned and de duplicated code prompts and is available in six sizes, ranging from huge (~ 200,000 lines equivalent to approx. 14.5M tokens), to micro (~ 6,200 lines and 2.4M tokens avg). Original data sourced from Vezora/Open Critic GPT, OpenCoder LLM/opc sft stage2, ise uiuc/Magicoder Evol Instruct 110K, and Multilingual Multimodal NLP/McEval Instruct File Language Lines : code huge English 200,000 code large English 100,000 code medium English 50,000 code small English 25,000 code tiny English 12,500 code micro English 6,250 Math calibration datasets This dataset consists of cleaned and de duplicated math prompts and is…
Runs entirely in your browser via DuckDB-Wasm — this dataset's real data file is loaded once, then queried locally. Nothing is sent to a server.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy