Tencent Releases Hy4 Preview, a 770B MoE Open-Weight Model With 1M Context

Tencent has released Hy4 preview

Tencent's Hy team released Hy4 preview on August 28, 2026 as an early open-weight flagship model aimed at long-horizon software engineering, office analysis, game development and scientific reasoning.

The model is a Mixture-of-Experts system with 770 billion total backbone parameters and 49 billion activated per token. Tencent lists a 1 million-token context window, 78 backbone layers, 256 routed experts per MoE layer plus one shared expert, and top-8 routed expert activation. A separate native multi-token-prediction layer is included for speculative decoding.

This is a released preview, not a future announcement: Tencent has published Hy4 preview and an FP8 variant, along with model weights, deployment guidance and fine-tuning resources. At the same time, Tencent explicitly labels the model an early version with known limitations and says further pre-training and post-training improvements are planned.

Architecture focuses on sparse attention and efficient serving

Hy4 preview uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse-index reuse. Tencent also uses identity Hyper-Connections in the residual pathway to expand information flow across layers.

The model's native MTP layer is designed to support speculative decoding. Tencent provides deployment recipes for both vLLM and SGLang, including OpenAI-compatible serving. The published examples enable tool-call parsing, reasoning parsing and MTP/speculative decoding for the FP8 checkpoint.

That makes this release relevant not only as a model-weight publication but also as an inference and deployment update: teams can evaluate the model in established open serving stacks rather than relying only on a proprietary hosted endpoint.

Tencent positions Hy4 around production-style work

Tencent says it built training data with internal domain experts including software engineers, game developers, finance analysts and security specialists. The company highlights four areas: long-horizon software engineering, office/document analysis, game prototyping and scientific research.

In a blind internal side-by-side evaluation, 163 Tencent experts rated outputs on 203 engineering tasks. Tencent reports Hy4 preview averaged 2.99 versus 2.92 for GLM 5.3 and 2.94 for Kimi K3, with slightly more wins than losses against both comparison models.

Those figures should be read as vendor-reported internal evaluation results, not independent proof of superiority. The task set was built around Tencent's own production workflows, which can be useful for product relevance but is not equivalent to a neutral third-party benchmark.

Open weights and Apache 2.0 licensing

Tencent says Hy4 preview and Hy4 preview-FP8 weights are available through multiple model-distribution channels, including Hugging Face, ModelScope, GitCode and CNB. The project's GitHub repository states that Hy4 preview is released under the Apache License 2.0.

The repository also includes a fine-tuning pipeline and points to AngelSlim for model compression and quantization. Because the model has 770B total parameters, practical self-hosting still requires substantial accelerator memory and multi-GPU infrastructure despite the MoE design activating a smaller subset of parameters per token.

What is released versus what is still evolving

Available now are the preview model weights, FP8 variant, deployment instructions, reasoning/tool-use parsing and fine-tuning resources.

Still evolving are the model's training recipe and behavior. Tencent specifically notes that Hy4 can spend longer than necessary reasoning through complex tasks and can over-verify its own work. The company says it plans to continue improving both pre-training and post-training.

That distinction matters: Hy4 preview is a real downloadable release, but it should not be treated as the final Hy4 generation.

Why Hy4 preview matters

The release adds another very large open-weight MoE system to the frontier-model ecosystem while combining long context, tool use, reasoning controls and open deployment paths.

For developers, the practical questions are how its quality, latency, memory requirements and tool behavior compare under their own workloads. Tencent's internal results are a useful starting point, but independent evaluation across coding, retrieval, long-context reliability and agentic workflows will be more informative than headline parameter counts alone.