Researchers have trained a new type of large language model (LLM) using GPUs spread across the globe, combining private and public data. This initiative suggests that the mainstream approach to building artificial intelligence may be disrupted.

The startup companies Flower AI and Vana, which pursue non-traditional AI building methods, collaborated to develop this new model, named Collective-1.

The technology developed by Flower allows the training process to be distributed across hundreds of computers connected via the internet. This technology has been used by some companies to train AI models without the need for centralized computing resources or data. Vana provides data sources, including private messages from platforms like X, Reddit, and Telegram.

Collective-1 is relatively small by modern standards, with 7 billion parameters—these parameters collectively endow the model with capabilities—whereas today’s most advanced models (such as those powering ChatGPT, Claude, and Gemini) have hundreds of billions of parameters.

Nic Lane, a computer scientist at the University of Cambridge and co-founder of Flower AI, stated that this distributed approach is expected to scale far beyond Collective-1. Lane added that Flower AI is training a 30 billion parameter model using conventional data and plans to train a 100 billion parameter model later this year—approaching the scale offered by industry leaders. "This could fundamentally change how people view AI, so we are going all out," Lane said. He also mentioned that the startup is incorporating images and audio into the training to create multimodal models.

Distributed model building could also shake up the power dynamics shaping the AI industry.

Currently, AI companies build models by combining vast amounts of training data with significant computing resources centralized in data centers. These data centers are filled with advanced GPUs and are interconnected via high-speed fiber optic cables. They also heavily rely on datasets created by scraping publicly available (though sometimes copyrighted) materials, such as websites and books.

This approach means that only the wealthiest companies and countries with large numbers of the most powerful chips can realistically develop the most powerful and valuable models. Even open-source models, such as Meta's Llama and DeepSeek's R1, are built by companies that own large data centers. The distributed approach could enable smaller companies and universities to build advanced AI by pooling homogeneous resources. Alternatively, it could allow countries lacking traditional infrastructure to build more powerful models by networking multiple data centers.

Lane believes that the AI industry will increasingly lean towards allowing training methods that break away from single data centers. "The distributed approach allows you to scale computing power in a more elegant way than data center models," he said.

Helen Toner, an AI governance expert at the Center for Emerging Technology Security, stated that Flower AI's approach is "interesting and potentially very relevant" to AI competition and governance. "It may be hard to keep up with the cutting edge, but it could be an interesting fast-follow approach," Toner said.

Divide and Conquer

Distributed AI training involves rethinking how computing is allocated to build powerful AI systems. Creating LLMs requires feeding large amounts of text into the model and adjusting its parameters to generate useful responses to prompts. Within a data center, the training process is divided so that parts of the task run on different GPUs, which are then periodically integrated into a single master model.

The new method allows work typically done within large data centers to be executed on hardware that may be miles apart, connected via relatively slow or unstable internet connections.

Some large companies are also exploring distributed learning. Last year, researchers at Google demonstrated a new scheme called DIstributed PAth COmposition (DiPaCo) for partitioning and integrating computations to make distributed learning more efficient.

To build Collective-1 and other LLMs, Lane collaborated with academic partners in the UK and China to develop a new tool called Photon, which makes distributed training more efficient. Lane stated that Photon improves upon Google's method by adopting a more efficient data representation and a scheme for sharing and integrating training. This process is slower than traditional training but more flexible, allowing for the addition of new hardware to accelerate training, Lane said.

Photon was co-developed by researchers from Beijing University of Posts and Telecommunications and Zhejiang University. The team released the tool last month under an open-source license, allowing anyone to use this method.

Vana, a partner of Flower AI in the effort to build Collective-1, is developing new methods for users to share personal data with AI builders. Vana's software allows users to contribute private data from platforms like X and Reddit to the training of large language models and potentially specify allowed end uses, even earning financial benefits from their contributions.

Vana co-founder Anna Kazlauskas stated that the idea is to make underutilized data available for AI training while giving users more control over how their information is used in AI. "This data is often not included in AI models because it is not public," Kazlauskas said, "This is the first time user-contributed data has been used to train foundational models, and users own the AI models created from their data."

Mirco Musolesi, a computer scientist at University College London, stated that a key benefit of distributed AI training methods may be that they unlock new types of data. "Scaling it to cutting-edge models will enable the AI industry to leverage vast amounts of decentralized and privacy-sensitive data, such as in healthcare and finance, for training without the risks of data centralization," he said.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。