AWQ Quant of nvidia/Llama-3.1-Nemotron-Nano-8B-v1
AWQ quant of nvidia/Llama-3.1-Nemotron-Nano-8B-v1 using llm-compressor for quantization.
Downloading quants with huggingface-cli
Click to view download instructions
Install hugginface-cli:
pip install -U "huggingface_hub[cli]"
Download quant by targeting the specific quant revision (branch):
huggingface-cli download ArtusDev/nvidia_Llama-3.1-Nemotron-Nano-8B-v1-AWQ