Se hizo una extraccion de MTP de un modelo 35b, formateo pesos. experimento 20 creo que elimine el template de los modelos el injerto MTP del otro modelo me gusto con este modelo , me falta recrear este modelo para agregar el template en algun momento lo borrare Uso (llama.cpp con MTP) bash Quants Quant Tamaño Q4 K M 20.5 GiB Q5 K M 23.9 GiB Q6 K 27.4 GiB Q8 0 35.2 GiB GPU 3090 24GB uso 22GB libre ~2GB modelo Q4 K M.gguf max tok compl tok time(s) tok/s medido tok/s nativo draft acc% finish 512 512 3.11 164.64 185.81154644562835 58.6% length 1024 1024 6.38 160.6 168.1821675659376 49.6% length 1536 1536 9.23 166.42 174.37983333556605 52.5% length 2048 2048 11.82 173.27 180.66270396669316 56.3% length 2560 2560 15.35 166.77 173.19459183649997 52.7% length 3072 3072 18.23 168.52 174.07312295115594 53.1% length 3584 3584 21.77 164.61 170.3211904082557 51.6% length 4096 4096 24.66 166.11 171.41060296776368 52.3% length 4608 4608 26.66 172.83 178.31869156802577 56.8% length 5120 5120 30.05 170.39 175.3896167291549 54.9% length 5632 5632 34.0 165.65 170.00963609304574 52.2% length 6144 6144 37.16 165.34 170.1135424832823 52.4% length 6656 6656 40.8 163.14 171.59782245044528 53.3% length 71…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy