AI Infrastructure: Where the next bottlenecks are emerging now

The focus is on High Bandwidth Memory (HBM). In this configuration, multiple DRAM chips are stacked and connected directly to the AI processor via wide data paths. As a result, HBM achieves significantly higher bandwidth than conventional DRAM memory. This is particularly crucial for large AI models. While processing a query, the model stores words and relationships that have already been processed in what is known as the KV cache. This cache prevents the need to completely recalculate the current context every time a new word is encountered. The longer the input, the more users working simultaneously, and the more frequently autonomous AI agents execute multiple steps, the greater the memory requirements become.

Deutsche Bank^[1] expects annual demand growth for HBM of around 40 percent by 2030, compared to 21 percent for standard DRAM. Morgan Stanley^[2] expects servers to account for 59 percent of DRAM demand by 2028. In 2023, that figure was 37 percent. For NAND, the share of enterprise SSDs is expected to rise from 18 to 65 percent.

Supply can only keep pace with this development at a slower rate. Building a new memory factory typically takes two to three years. In addition, one bit of HBM requires about three times as much wafer area—that is, space—as one bit of conventional DRAM. For HBM4 and HBM4E, which are expected to be available starting in the second half of 2026, this figure could even be four times as much. This additional space requirement comes at the expense of other applications, such as in computers, smartphones, or cars. For the second quarter of 2026, price increases of 58 to 63 percent quarter-over-quarter were expected for standard DRAM.

For NAND memory, expectations were even higher, with price increases of 70 to 75 percent compared to the previous quarter. On the one hand, this indicates strong pricing power; on the other hand, it also highlights the potential for a sharp decline as soon as supply grows faster than demand. The market is highly concentrated. SK Hynix, Samsung, and Micron together control more than 90 percent of the DRAM market.

Micron's $200 billion program in the U.S. is scheduled to run for 20 years. On average, this amounts to approximately $10 billion per year. $150 billion is earmarked for manufacturing and $50 billion for research. Plans include plants in Idaho and New York, the modernization of the Virginia site, and capacity for HBM packaging. Samsung spent approximately $40 billion on capital expenditures in 2025 and plans to spend more than $73 billion on capital expenditures and research in 2026, including in other business areas. SK Hynix approved investments of approximately $38 billion for two memory factories. In addition, Samsung, SK Hynix, and suppliers announced a South Korean AI and semiconductor program totaling approximately $518 billion. The plan includes four factories and an HBM packaging cluster. The timeline has not yet been specified.

In the short term, memory will remain in short supply. In the medium term, however, the risk of a new supply cycle is increasing. Competition is also intensifying in the NAND market. While a Japanese NAND specialist is benefiting from rising demand for enterprise SSDs, it is hardly affected by the HBM boom. At the same time, Chinese suppliers are gaining market share. In the second quarter of 2026, for example, a Chinese competitor accounted for approximately 14 percent of global NAND shipments by bit, thereby surpassing an established supplier in terms of volume. In the high-end enterprise product segment, however, it still lags behind. Competition from Chinese providers is also intensifying in the DRAM market. In the short term, AI demand is providing a tailwind for NAND. In the long term, however, growing competition could increase price pressure.

At the same time, technological progress could reduce the memory requirements per AI query. According to published tests, Google’s TurboQuant compression method can reduce the KV cache without any measurable loss of accuracy. This reduces the computational memory requirements per query, which weighed on memory stocks in 2026. This is countered by the so-called Jevons effect. As the cost of using AI decreases, usage itself may increase. A lower memory requirement per query therefore does not necessarily lead to lower overall demand.

In high-performance AI chips, the IC substrate serves as the link between the silicon and the printed circuit board. It thus connects the processor-memory package to the system. No modern chip package can function without a substrate. Solutions featuring HBM and multiple chiplets fall under the category of advanced packaging. High-performance processors primarily use ABF substrates. The insulating material enables particularly fine conductive traces. However, larger chip packages and more HBM increase space requirements and manufacturing costs.

Goldman Sachs^[3] estimates that NVIDIA’s upcoming Rubin GPU will require 123 percent more ABF area than Blackwell. For the new Vera CPU compared to Grace, the figure is 47 percent. New server processors could require more than 30 percent additional substrate area. According to Goldman Sachs, the ABF market could grow from $7.7 billion in 2025 to $33 billion in 2028. At the same time, Goldman Sachs projects a 51 percent supply shortfall. In early 2026, lead times rose from three to four months to more than twelve months. New capacity is not expected to provide significant relief until 2028 or 2029. Long construction and qualification times give established suppliers an advantage.

Two suppliers lead the market with market shares of approximately 21 and 20 percent, respectively. One of them has a broad product portfolio in AI chips and server processors. More than 70 percent of its ABF shipments are covered by long-term contracts. This stabilizes capacity utilization and revenue, but at the same time limits the impact of higher spot prices on revenue. Another supplier is more exposed to market prices. This increases margin potential but also carries a higher risk of setbacks.

Substrate production does not depend solely on production lines. As power density increases, T-Glass is becoming more important. This is a heat-resistant glass fabric that stabilizes fine substrate structures. Shortages of this material could slow the expansion of ABF capacities. A European supplier has announced plans to invest between 1.5 and 2 billion euros in an AI project. However, high investment levels, critical materials, and delays continue to pose risks.

Modern AI models distribute their work across thousands of accelerators. When the network slows down, even the graphics processing units (GPUs) cannot reach their full potential. At 800 gigabits, 1.6 terabits, and later 3.2 terabits per second, copper is increasingly reaching its physical limits. Electrical signals lose quality over longer distances, generate heat, and consume more energy. Fiber optics, on the other hand, transmit data as light with high bandwidth over greater distances and consume less energy per bit. For short distances, copper remains more cost-effective. Until now, the motto has therefore been: “Copper, if possible. Fiber, if necessary.” However, as power density increases, this balance is shifting more and more in favor of optical connections.

For example, an AI rack may require 16 to 36 times more fiber-optic cable than a traditional rack. With Co-Packaged Optics (CPO), the optical components are located directly next to the network chip. This shortens the electrical paths, which—at 1.6 terabits per second—can reduce energy consumption from about 30 to nine watts. Globally, CPO solutions currently generate less than one billion U.S. dollars in revenue. However, a study projects revenue of approximately 15 billion U.S. dollars by 2030. Customer approvals, quality, and cost-effectiveness remain critical factors.

A leading supplier of optical components estimates its market share in so-called EML lasers at 50 to 60 percent. These lasers convert electrical data signals into light signals that are transmitted via optical fibers. In the June quarter of 2026, this supplier’s revenue rose by 109 percent to $1.006 billion. The gross margin reached 50.4 percent and the operating margin 36.6 percent. According to management, demand for EML lasers exceeded supply by more than 30 percent. Another supplier, which covers a broader range of lasers—including optical switches—does not expect broader CPO shipments until 2028.

Chinese suppliers are also benefiting from the rising demand for optical modules. A leading manufacturer generated revenue of $5.3 billion in 2025. In the first quarter of 2026, revenue rose by 192 percent to $2.7 billion. Profit increased by 274 percent to $0.9 billion. This underscores the high demand for 800G and 1.6T connections. At the same time, dependence on individual markets and major customers is becoming apparent: 61.7 percent of revenue came from the U.S., and Google is among the largest customers.

Added to this are valuation and geopolitical risks. For example, the U.S. government recently considered import restrictions on optical components from China. In addition, suppliers are expanding their capacities. It is therefore crucial whether demand can keep pace with the rising supply and whether high margins can be maintained.

For memory chips, HBM contracts, wafer starts, yields, and capacity allocation are critical. Wafer starts provide early indications of future supply. For substrates, delivery times, prices, and customer contracts are important. For optical interconnects, the transition from 800G to 1.6T, CPO customer approvals, and customer concentration are critical. Warning signs include duplicate orders, falling delivery times, delayed data centers, or new factories without corresponding purchase agreements. The key question remains whether companies can sustainably translate this shortage into free cash flow and high returns on investment.