Overview HyperCLOVA X SEED 14B Think is a next generation language model that moves beyond the conventional approach of simply increasing model size to improve performance. It combines HyperCLOVA X’s lightweighting technology for building high efficiency LLMs with advanced reasoning capabilities. Its development relied on two key technologies: (1) Pruning & Knowledge Distillation, which achieves both compactness and high performance, and (2) a Reinforcement Learning (RL) pipeline, which maximizes reasoning ability. By pruning low importance parameters and distilling knowledge from a large model into a smaller one, training costs have been significantly reduced. On top of this, the latest RL recipe validated in HyperCLOVA X Think is applied in a multi stage process: (1) Supervised Fine Tuning (SFT), (2) Reinforcement Learning with Verifiable Rewards (RLVR), (3) Length Controllability (LC) for reasoning path optimization, and (4) a joint training of Reinforcement Learning from Human Feedback (RLHF) and RLVR. It is a considerable challenge to equip a pruned, knowledge distilled model with reasoning capabilities, since reductions in training costs and model size often degrade reasoning…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy