Source: Wall Street News
Every layer of AI infrastructure is simultaneously under pressure, with opportunities and misjudgments coexisting.
Dylan Patel, founder of SemiAnalysis, recently accepted a podcast interview, systematically outlining the core dynamics and investment logic of the current AI infrastructure stack.
His judgments cover model economics, the memory supercycle, CPU repricing, timelines for CPO, and structural opportunities in data center energy supply.
In response to the market's general skepticism about AI return on investment (ROI), Dylan revealed that Anthropic has achieved positive free cash flow in the second quarter of this year, with annual recurring revenue exceeding $50 billion, gross margin exceeding 70%. On the enterprise side, the productivity leap brought by the latest AI models far exceeds the increase in computing costs, prompting companies to cut other software expenses to sustain their explosive AI budgets.
In terms of hardware evolution, the paradigm shift towards inference models is reconstructing market demand.
Dylan emphasized, storage faces structural shortages lasting years, with still 2 to 3 times upside potential; meanwhile, although agents and reinforcement learning have boosted CPU demand, the seller's market has priced this too high, the growth of CPUs mainly comes from historical "catch-up," and their absolute value in AI servers is still far less than GPUs.
Dylan believes that the eagerly anticipated large-scale rollout of Co-Packaged Optics (CPO) has been explicitly postponed to late 2028 to 2029, unexpectedly extending the bonus period for copper connectors. The constraints in the power grid for transmission and distribution are pushing data centers towards "behind the meter" (self-built power), creating huge industrial energy and power conversion supply chain investment opportunities beyond traditional chip investments.
Anthropic Takes the Lead, AI Demand Narrative Begins to Materialize
In response to the skepticism regarding AI companies' ROI in the market, Dylan Patel provided specific data.
"Anthropic has achieved positive free cash flow in the second quarter, profitable in April, profitable in May, and looks likely to be the same in June." He stated that Anthropic's annual recurring revenue has exceeded $50 billion, with a gross margin exceeding 70%. OpenAI's revenue has also rapidly grown with the increasing adoption of Codex.
SemiAnalysis's own spending trajectory also confirms this trend. Last November, the company's 90-person team had an annual AI expenditure of less than $100,000; by the end of January this year, due to the large-scale rollout of Claude Code, this figure soared to $4 million annually; it has now reached $11 million, peaking at an annualized figure of $14 million at one point."The human costs combined with AI costs have already exceeded one-third; it is likely to reach half by the end of the year."
He also pointed out that newer, stronger models do not necessarily come at a higher price in practical use. Old models might require 100,000 tokens and 10 interactions to complete a task, whereas new models might only need 25,000 tokens and 1 interaction. "Every time a model upgrades from 4.6 Opus to 4.7 Opus, our spending first drops for a week, then surges—because everyone sees that things that couldn't be done before can now be accomplished."
He believes this is also one of the core reasons why Anthropic holds an advantage in competition with OpenAI: higher token efficiency and lower overall costs for users.
Memory: Structural Shortage, Not Ordinary Cycle
Among all hardware categories, Dylan Patel is most confident in his judgment of memory.
"This is not a short-term shortage, but a structural shortage that will last for years." He pointed out that memory capacity grows by only 20% to 30% each year, while AI-side demand is doubling upon doubling, and the gap between the two will continue to widen.
The core logic driving this judgment comes from the impact of inference models on KV caching. Traditional dialog-based reasoning has context lengths measured in thousands of tokens, consuming limited KV cache; however, with the emergence of inference models represented by o1, the context length has explosively increased, causing KV cache to expand dramatically, making memory the most directly benefited category. SemiAnalysis will publish a report in December 2024 specifically pointing out this trend.
Rigid constraints on the supply side will force downstream markets to reallocate limited memory resources. He predicts that consumer electronics, which have low price elasticity, will be the first to face pressure—mid-range and low-end smartphone manufacturers have already seen a 40% decline in shipment volume, and the prices of iPhones and MacBooks will rise next year. "Memory will continue to increase in price, and consumer electronics will be compressed to a new level until AI obtains the memory it needs to be truly sufficient."
He added that even if a downturn in the cycle comes at that time, "from trough to trough, long-term growth is undoubtedly there."
CPU: Limited Catch-up Market, Don't Over-Extrapolate
The CPU has become a new protagonist in this year's AI infrastructure narrative, but Dylan Patel holds a clear cautionary stance on this.
The logic behind the recovery of CPU demand is clear: reinforcement learning requires a large number of CPUs to run environment validation (code unit testing, simulation, etc.); agent reasoning requires frequent model calls to tools and interaction with the real world, which heavily relies on CPU computing power.
At the same time, in recent years there has been a large-scale shipment of AI chips, but the corresponding CPUs are seriously insufficient, and we are currently in a concentrated catch-up phase, with ARM, Intel, and AMD all benefiting, and Nvidia's Vera CPU also providing a $20 billion revenue guidance.
"But I want to give an important warning: there is a lot of catch-up effect here." He stated that once the historical backlog is completed, only incremental demand remains and demand will return to normal. In absolute amounts, Blackwell is about $50,000 per unit, while CPUs are about $5,000; even if CPUs are allocated more in ratio, the dollar amount is still far less than AI acceleration chips.
"Memory and AI acceleration chips are the main components; CPU has been revaluated after being undervalued and is now priced more reasonably, but it will not continue to grow at a rate exceeding that of AI chips indefinitely."
Optical Interconnect: Optimistic Long-Term, Cautious in Short to Medium Term on CPO
Networking and optical interconnects is another area where market sentiment is high, but Dylan Patel takes a cautious attitude towards the pace of CPO (Co-Packaged Optics) rollout.
"I judge that CPO's large-scale mass production will occur around late 2028 to 2029." He pointed out that the current manufacturing yield, chip design, and supply chain maturity have not yet reached the standards for large-scale deployment, while Nvidia's Rubin and its subsequent architecture Feynman will continue to use a full copper solution, and CPO on the GPU side will need to wait for several generations of chip iterations.
He revealed that SemiAnalysis just released a report to institutional subscription clients last week, anticipating a more positive outlook for copper cables and non-CPO optical solutions in the short to medium term, while holding a cautious stance on CPO. Some design changes of downstream chips (such as Rubin Ultra's Kyber removing the 800V design) have further delayed CPO rollout. Companies producing copper connectors, such as Amphenol, will benefit more than expected as a result.
"CPO will happen in the long term, and copper cables will eventually be replaced, but the timeline has been pushed back; in the short to medium term, copper cables still have a lot of opportunities."
Power: Self-Built Power Supply Will Become Mainstream, Diverse Innovation Paths
The power supply of data centers is becoming the hardest physical constraint for AI growth.
According to Dylan Patel's predictions, the new power consumption for data centers will increase by 20 gigawatts this year, 30 gigawatts next year, and 50 gigawatts the year after, growing almost explosively.
He breaks down the energy problem into three dimensions: transmission, generation, and conversion. Transmission is the most difficult link to break through, involving regulatory policies, the monopoly structure of local power companies, and cost-sharing mechanisms, which are unlikely to change in the short term. Generation and conversion present broad opportunities.
He predicts that in the coming years, half of the new power consumption in data centers will come from "behind the meter" (self-built power), rather than relying on the public power grid.
The current mainstream solution is combined cycle gas turbines (CCGT), from manufacturers like GE Vernova, Mitsubishi, and Siemens; at the same time, there are also non-traditional solutions such as reciprocating engines, industrial gas turbines, and even retrofitted engines from ships, trains, and trucks. "It sounds rough, but it runs, and it has already been used."
In the longer term, he judges that in about two years, the comprehensive costs of solar energy combined with storage will be lower than gas generation; further on, there are space data centers—deploying computing chips in orbit, where solar panels do not need to penetrate the atmosphere, yield much higher energy density than on the ground and do not require storage.
The conversion side is also full of investment opportunities, from IGBT, silicon carbide to gallium nitride MOSFETs, and solid-state transformers, UPS, and supercapacitors, the entire voltage conversion chain is rapidly evolving.
Currently, the largest research department at SemiAnalysis is no longer semiconductors, but a team internally known as "DEI" (Data Center, Energy, and Industry), tracking the deployment dynamics of every data center and power plant globally.
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
