Qwen3 30B A3B AWQ Uploaded by Eric Hartford Copied from Modelscope https://www.modelscope.cn/models/swift/Qwen3 30B A3B AWQ Original model https://huggingface.co/Qwen/Qwen3 30B A3B Modelscope AWQ Modelcard: import torch from modelscope import AutoModelForCausalLM, AutoTokenizer model name = "swift/Qwen3 30B A3B AWQ" load the tokenizer and the model tokenizer = AutoTokenizer.from pretrained(model name) model = AutoModelForCausalLM.from pretrained( model name, torch dtype=torch.float16, device map="auto" ) prepare the model input prompt = "Give me a short introduction to large language model." messages = [ {"role": "user", "content": prompt} ] text = tokenizer.apply chat template( messages, tokenize=False, add generation prompt=True, enable thinking=True Switches between thinking and non thinking modes. Default is True. ) model inputs = tokenizer([text], return tensors="pt").to(model.device) conduct text completion generated ids = model.generate( model inputs, max new tokens=32768 ) output ids = generated ids[0][len(model inputs.input ids[0]):].tolist() parsing thinking content try: rindex finding 151668 ( ) index = len(output ids) output ids[:: 1].index(151668) except ValueError: in…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy