Author｜Zhou Yixiao
WeChat｜smiletalker

Computer graphics researcher Tong Xin recently joined AI 3D generation company Meshy as Chief Scientist.

After obtaining his Ph.D. from Tsinghua University in 1999, Tong Xin joined Microsoft Research Asia, where he worked for 25 years, leading the web graphics group and serving as Partner Research Manager. His research areas span texture synthesis, material modeling, realistic rendering, 3D animation, and geometric processing, making him one of the most senior researchers in the graphics team at Microsoft Research Asia.

His work extends beyond academic papers. The introduction by the China Computer Society mentions that Tong Xin led his team in developing technologies such as high dynamic range texture compression, GPU virtualization, 3D printing support, and geometric detail rendering, some of which later entered the Xbox game development API, Windows 3D printing drivers, and Direct3D graphics development toolkits.

Tong Xin has published more than 190 papers, with over 60 published in top graphics conferences and journals like SIGGRAPH and ACM TOG. In 2018, he received the Outstanding Award in Computer Graphics from the China Computer Society; in 2024, he received the AsiaGraphics Outstanding Technical Contributions Award, which is awarded to a maximum of one person each year to recognize researchers who make outstanding technical contributions in the field of computer graphics.

After leaving Microsoft in 2024, Tong Xin joined Anuttacon to continue research on AI and game graphics. Now, he has gone to Meshy.

At this year's GDC, Meshy's game studio, Meshy Game Studio, showcased the AI native game "Black Box: Infinite Arsenal." The recently unveiled Mora integrates Coding Agent, 3D generation, and real-time video models into a single system, hoping that the mini-games created by AI will have clear gameplay and spatial structures while also generating richer audiovisual expressions in real time.

1

Utilizing generated 3D effectively

Meshy launched its AI 3D products in 2023. By July this year, the company disclosed that it had over 12 million registered users and generated more than 100 million 3D models, with an ARR approximately 12 times that of a year ago, and completed nearly $400 million in Series B financing with a valuation of $1.5 billion.

These models have been utilized in gaming, 3D printing, design, and product development. Clients and users listed on Meshy's official website include game companies like Nexon, NetEase Games, and 37 Interactive Entertainment, as well as 3D printing and manufacturing brands like Tuozhu, Chuangxiang 3D, ELEGOO, Flashforge, and xTool.

The product itself is also addressing increasingly specific problems. AI can already create a 3D model, but once it enters the production process, it still needs to check whether the shape is accurate, whether components can be separated, whether the topology is clean, and whether it is easy to continue editing in the future.

Released in August, Meshy 7 focuses on geometry alignment, which is the proper alignment of generated 3D shapes with reference images. Simply put, even if a mechanical part looks quite similar, if there's a scale deviation, if two adjacent parts are stuck together, or if the groove in the reference image only exists in the texture without being integrated into the geometric structure, the following animation, editing, and manufacturing will be affected.

To address this, Meshy has developed an automated geometric evaluation system, breaking down results into overall scale, spatial distribution, and surface detail as three metrics. In Meshy's built single-view benchmark, Meshy 7 achieved 81.0%, 79.7%, and 59.8% in these three categories, outperforming other tested models in the same evaluation.

Smart Topology handles later stages of production. It can directly generate 3D models with native components and control the number of model faces, reducing manual decimation, retopology, and component organization.

When discussing AIGC in 2023, Tong Xin referred to similar issues as the “last mile” of 3D generation.

His meaning is not complicated. Generated models can complete the first step of content production, but a truly actionable 3D model for animation and interaction still requires information such as semantics, component structure, and surface properties. Traditional graphics have established a mature production process for these tasks, while generative AI still needs to handle parts after generation.

Three years later, consistency, topology, and component separation have become very practical metrics in AI 3D products. Meshy has also gained users, clients, and revenue through this product.

This year, Meshy started talking more about “AI for Fun.” Meshy founder and CEO Hu Yuanming explained this by saying that they hope AI will ultimately change not just how content is produced, but also how people play and what they experience. Meshy summarizes this as “bringing joy and meaning to humanity through AI.” Mora is currently the most complete technical attempt in this line of thinking.

1

Separately developing rules, space, and visuals

In January this year, Google was the first to open Project Genie to US AI Ultra users. Users can input text and images to create an environment generated in real-time by Genie 3, allowing them to move and explore within it.

Such products easily lead to thoughts of a more radical approach. Since video models can now generate and maintain an explorable space simultaneously, could a model in the future create a full game directly?

Hu Yuanming recently published an article on his official account titled "Mora: Beyond 'World Models'," specifically discussing this issue.

He listed eight limitations of existing interactive video models, including the difficulty of maintaining spatial consistency over time, the complexity of handling physics and precise game logic, limited interaction modes, and issues with duration and operational latency. A world generated by a video model may look very realistic, but “looking like it exists” and “actually running according to a stable set of rules” are two different things.

Mora breaks down these tasks for handling.

Coding Agent first generates the skeleton of the game world, running code, and gameplay logic, while Meshy’s 3D generation supplements scenes and assets, providing 3D structures and control information to the real-time video model. The video model is responsible for the final visuals and audio. The code and 3D provide clearer operational rules and spatial structures, reducing the challenge of having the video model maintain the entire world on its own.

A relatively straightforward example is the demo similar to "Portal" created by Mora. "Portal" is Valve's classic spatial puzzle game where players change the paths of characters and objects through two connected portals, relying heavily on stable spatial relationships and precise rules.

Mora's Portal-like Demo

In Mora's demo, the scene structure and puzzle logic are maintained by code and the 3D system, while players are responsible for solving puzzles, and the real-time video model generates the final visuals seen. Hu Yuanming also directly mentioned the current limitations in the article; the model is still unable to accurately represent the content within the portals, so they have temporarily turned off the rendering of the portals themselves.

Other demos include multiplayer platform jumping, racing, planting, building, and mini-games with physics collisions. Some scenes can switch visual styles directly during runtime. For the mini-games generated quickly by Coding Agent, real-time video models can provide richer, adaptable visual expressions.

Hu Yuanming has set a clear goal for Mora, hoping to “ensure that the mini-games generated by Coding Agent have AAA-level audiovisual experiences.”

The company has accumulated experience in products, graphics, and large model training in 3D generation. Mora further combines these capabilities with Coding Agent, video generation, and real-time inference systems to explore richer real-time interactive content.

This approach also connects with the changes in Tong Xin's research focus over the years.

1

The production processes of graphics are also changing

Tong Xin has not recently started introducing machine learning into 3D graphics.

In 2017, his work on O-CNN was published at SIGGRAPH. Traditional convolutional neural networks excel at processing regularly arranged 2D images, but 3D models are typically composed of irregular meshes or point clouds; directly slicing high-resolution 3D space into dense voxels rapidly increases computational and memory overhead. O-CNN organizes 3D data using octrees, performing convolution computations only near the object surfaces, allowing the neural network to process high-resolution 3D shapes more efficiently.

The key issue then was how to enable deep learning to better understand 3D.

In recent years with RenderFormer, models have begun to take on another core task within the graphics workflow.

3D rendering requires calculating the final visuals based on geometry, materials, and lighting in the scene. RenderFormer retains triangular meshes and material data from 3D scene information, allowing a transformer to learn the lighting relationships within the scene, generating the final image with global illumination effects. This research will be published at SIGGRAPH 2025, with Tong Xin as one of the authors of the paper.

TRELLIS addresses another issue: how to enable generative models to represent 3D content more flexibly. It learns a unified internal 3D representation before outputting different forms of 3D results according to needs. The LPM 1.0 released this year steps into character performance, attempting to learn how characters speak, move, and react directly from videos, reducing reliance on traditional character rigging and animation production workflows.

This also connects back to another path Tong Xin mentioned in 2023.

He calls it the “starting kilometer.” After the advent of new representations and generative models, should 3D content still adhere to the original production methods, or can a new workflow be established?

RenderFormer and Mora handle different tasks, but there is a specific similarity. Both retain explicit 3D scene information, allowing models to generate visuals based on this information. Tong Xin's recent research on 3D representation, neural rendering, and video characters has also shown more intersections with the issues Meshy is exploring.

As for the specific research Tong Xin will oversee after joining, Meshy has not disclosed further details.

1

Moving beyond just generating 3D models

Mora 1 currently remains an early architecture validation.

The limitations self-listed by Meshy include that multiplayer visuals can easily become blurry, and small character faces still exhibit typical video generation errors. According to Hu Yuanming, including network latency, current Mora has a delay of approximately 300 milliseconds from player input to visual feedback, and the team hopes to further reduce this to under 100 milliseconds. This is still crucial for games like FPS that require quick perspective shifts.

Mora 2 is already in development, and the non-open creative system will also be reserved for future versions. The team’s recruitment posts have already covered video foundation models, large-scale training, real-time generation, agents, and system engineering.

Three years ago, Tong Xin mentioned that generative AI left graphics with the “final mile” and the “starting kilometer.” The former involves enabling generated 3D to truly enter production, while the latter requires rethinking how 3D content should be approached.

Now, Meshy is working on both fronts.

When Tong Xin joined, the company’s goal had already expanded beyond simply enabling AI to produce a 3D model faster. It is also attempting something else: to combine these models with code and video generation to create a game world that can run, interact, and has sufficiently rich audiovisual expressions.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。