HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

English | 简体中文

"What Is Now Proved Was Once Only Imagined"

🎥 Video

https://github.com/user-attachments/assets/b56f4750-25c9-48fb-83ff-d58526711463

🔥 News

• [May 18, 2026]: 🤗 Open-source World Generation inference code and WorldStereo 2.0 model weights!
• [May 11, 2026]: 🤗 Open-source HY-Pano 2.0 inference code and model weights!
• [April 16, 2026]: 🚀 Release HY-World 2.0 technical report & partial codes!
• [April 16, 2026]: 🤗 Open-source WorldMirror 2.0 inference code and model weights!

📋 Table of Contents

• 📖 Introduction
• ✨ Highlights
• 🧩 Architecture
• 📝 Open-Source Plan
• 🎁 Model Zoo
• 🤗 Get Started
• 🔮 Performance
• 🎬 More Examples
• 📚 Citation

📖 Introduction

HY-World 2.0 is a multi-modal world model framework for world generation and world reconstruction. It accepts diverse input modalities — text, single-view images, multi-view images, and videos — and produces 3D world representations (meshes / Gaussian Splattings). It offers two core capabilities:

• World Generation (text / single image → 3D world): syntheses high-fidelity, navigable 3D scenes through a four-stage method —— a) with HY-Pano 2.0, b) with WorldNav, c) with WorldStereo 2.0, and d) with WorldMirror 2.0 & 3DGS learning.
• World Reconstruction (multi-view images / video → 3D): Powered by WorldMirror 2.0, a unified feed-forward model that simultaneously predicts depth, surface normals, camera parameters, 3D point clouds, and 3DGS attributes in a single forward pass.

HY-World 2.0 is an open-source state-of-the-art world model. We released all model weights, code, and technical details to facilitate reproducibility and advance research in this field.

Why 3D World Models?

Existing world models, such as Genie 3, Cosmos, and HY-World 1.5 (WorldPlay+WorldCompass), generate pixel-level videos — essentially "watching a movie" that vanishes once playback ends. HY-World 2.0 takes a fundamentally different approach: it directly produces editable, persistent 3D assets (meshes / 3DGS) that can be imported into game engines like Blender/Unity/Unreal Engine/Isaac Sim — more like "building a playable game" than recording a clip. This paradigm shift natively resolves many long-standing pain points of video world models:

All above are real 3D assets (not generated videos) and entirely created by HY-World 2.0 -- captured from live real-time interaction.

✨ Highlights

• Real 3D Worlds, Not Just Videos

  Unlike video-only world models (e.g., Genie 3, HY World 1.5), HY-World 2.0 generates real 3D assets — 3DGS, meshes, and point clouds — that are freely explorable, editable, and directly importable into Unity / Unreal Engine / Isaac. From a single text prompt or image, create navigable 3D worlds with diverse styles: realistic, cartoon, game, and more.

• Instant 3D Reconstruction from Photos & Videos

  Powered by WorldMirror 2.0, a unified feed-forward model that predicts dense point clouds, depth maps, surface normals, camera parameters, and 3DGS from multi-view images or casual videos in a single forward pass. Supports flexible-resolution inference (50K–500K pixels) with SOTA accuracy. Capture a video, get a digital twin.

• Interactive Character Exploration

  Go beyond viewing — play inside your generated worlds. HY-World 2.0 supports first-person navigation and third-person character mode, enabling users to freely explore AI-generated streets, buildings, and landscapes with physics-based collision. Go to our product page for free try.

🧩 Architecture

• Refer to our tech report for more details

  A systematic pipeline of HY-World 2.0 — Panorama Generation (HY-Pano-2.0) → Trajectory Planning (WorldNav) → World Expansion (WorldStereo 2.0) → World Composition (WorldMirror 2.0 + Splattings Learning) — that automatically transforms text or a single image into a high-fidelity, navigable 3D world (3DGS/mesh outputs).

📝 Open-Source Plan

• [x] Technical Report
• [x] WorldMirror 2.0 Code & Model Checkpoints
• [x] Full Inference Code for World Generation (WorldNav + WorldStereo + World Composition)
• [x] Panorama Generation (HY-Pano 2.0) Model & Code
• [x] World Expansion (WorldStereo 2.0) Model & Code

🎁 Model Zoo

World Reconstruction — WorldMirror Series

Panorama Generation — HY-Pano Series

World Expansion — WorldStereo Series

We recommend referring to our previous works, WorldStereo and WorldMirror, for background knowledge on 3D world generation and reconstruction.

🤗 Get Started

Install Requirements

We recommend CUDA 12.8 and Python 3.11+. The easiest path is to prepare one shared environment, first make World Reconstruction (WorldMirror 2.0) work, and then install the extra components required by World Generation.

1. Create the shared environment

git clone https://github.com/Tencent-Hunyuan/HY-World-2.0
cd HY-World-2.0

conda create -n hyworld2 python=3.11.15
conda activate hyworld2

2. Install World Reconstruction dependencies

After this step, the environment is ready for worldrecon / WorldMirror 2.0.

# Base dependencies shared by worldrecon and worldgen
pip install -r requirements.txt

# Recommended: install the custom gsplat variant once for both worldrecon and worldgen
cd hyworld2/worldgen/third_party/gsplat_maskgaussian
pip install -e . --no-build-isolation
cd ../../../../

If you only need worldrecon and want a simpler fallback, official gsplat is also supported:

pip install git+https://github.com/nerfstudio-project/gsplat.git

Install one FlashAttention backend:

# Recommended for Hopper GPUs: FlashAttention-3
git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention/hopper
python setup.py install
cd ../../
rm -rf flash-attention

# Simpler alternative: FlashAttention-2
pip install flash-attn --no-build-isolation

3. Add extra World Generation dependencies

Run the following extra steps only if you need worldgen. These commands assume the shared hyworld2 environment above is already active.

# Git-based dependencies require torch/CUDA to be installed first
pip install --no-build-isolation -r requirements_git.txt

# recastnavigation is managed as a git submodule
git submodule update --init --recursive

# Recast navmesh extension for trajectory planning
cd hyworld2/worldgen/third_party/navmesh
pip install . --no-build-isolation
cd ../../../../

For HY-Pano-2 installation, please refer to hyworld2/panogen/README.md.

Code Usage — Panorama Generation (HY-Pano-2)

For full documentation and CLI reference, see hyworld2/panogen/README.md.

We provide a diffusers-like Python API for HY-Pano 2.0. Model weights are automatically downloaded from Hugging Face on first run.

from pipeline import HunyuanPanoPipeline

pipeline = HunyuanPanoPipeline.from_pretrained('tencent/HY-World-2.0')
output = pipeline('input.png')
output.save('output_panorama.png')

Code Usage — World Generation (WorldNav, WorldStereo-2, and 3DGS)

The world Generation pipeline turns a panorama scene into a navigable 3D world through five stages:

For full documentation, prerequisites, and CLI arguments, see hyworld2/worldgen/README.md.

Code Usage — WorldMirror 2.0

WorldMirror 2.0 supports the following usage modes:

• Code Usage
• Gradio App

We provide a diffusers-like Python API for WorldMirror 2.0. Model weights are automatically downloaded from Hugging Face on first run.

from hyworld2.worldrecon.pipeline import WorldMirrorPipeline

pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
result = pipeline('path/to/images')

With Prior Injection (Camera & Depth):

result = pipeline(
    'path/to/images',
    prior_cam_path='path/to/prior_camera.json',
    prior_depth_path='path/to/prior_depth/',
)

For the detailed structure of camera/depth priors and how to prepare them, see Prior Preparation Guide.

CLI:

# Single GPU
python -m hyworld2.worldrecon.pipeline --input_path path/to/images

# Multi-GPU
torchrun --nproc_per_node=2 -m hyworld2.worldrecon.pipeline \
    --input_path path/to/images \
    --use_fsdp --enable_bf16

Important: In multi-GPU mode, the number of input images must be >= the number of GPUs. For example, with --nproc_per_node=8, provide at least 8 images.

Gradio App — WorldMirror 2.0

We provide an interactive Gradio web demo for WorldMirror 2.0. Upload images or videos and visualize 3DGS, point clouds, depth maps, normal maps, and camera parameters in your browser.

# Single GPU
python -m hyworld2.worldrecon.gradio_app

# Multi-GPU
torchrun --nproc_per_node=2 -m hyworld2.worldrecon.gradio_app \
    --use_fsdp --enable_bf16

For the full list of Gradio app arguments (port, share, local checkpoints, etc.), see DOCUMENTATION.md.

🔮 Performance

Fo