Tencent Releases Hy4 Model with 770 Billion Parameters and 1 Million Context

Tencent has launched the Hy4 preview, a new model that adopts the MoE architecture with a total of 770 billion parameters, activating 49 billion at a time, and extending the context to 1 million. Compared to Hy3, which has 295 billion total parameters, 21 billion activation parameters, and a context of 256k, Hy4 has doubled in scale and increased the long context by approximately four times. Hy4 emphasizes its application in real tasks such as coding, office work, gaming, and scientific research, capable of building a Three.js 3D website from scratch, creating playable game demos in Unity, processing 72 financial documents, and checking for duplicate reimbursements, exceeded limits, and budget anomalies. Hy4 has also been trained and iterated in conjunction with Tencent products like WorkBuddy and CodeBuddy. Hy4 participates in its own development, helping to optimize training methods, data strategies, evaluation systems, and underlying operators, forming a self-improvement loop. The Hy4 preview has been integrated into multiple Tencent products, and Tencent has indicated in its financial report that Hy4 will be larger than Hy3, with real product feedback serving as an important source for model training.