AI learns to grow deep, not wide

news23092026

The Industry’s 90-Degree Turn: From Horizontal Scale to Vertical Architectural Complexity

Instead of building larger monoliths, engineers have begun mastering volume. The industry is executing a 90-degree pivot, replacing the horizontal race for scale with vertical architectural complexity.

This shift unites two recent technological events: Alibaba’s software announcement (the Qwen4 family) and semiconductor research from the University of Hong Kong. Both language models and new microchips are abandoning whole-system operation for a single task. Just as software learns to selectively activate only the necessary experts within a trillion-parameter network, hardware shifts computation into multi-layered three-dimensional structures, minimizing idle energy loss. Analyzing these parallels reveals the birth of a new engineering paradigm where efficiency is defined not by core count or model weight, but by the system’s ability to dynamically manage space—both logical and physical.

Efficiency vs. Scale Through Spatial Reorganization

The classic paradigm of technological progress, which required linear resource expansion to reach new heights, has exhausted itself. In software, this manifested as endless neural network enlargement; in hardware, it was the attempt to endlessly compress transistors onto a silicon plane. Both paths hit a dead end of exorbitant electricity bills and thermodynamic laws that prevent heat dissipation from super-dense structures.

The «Computational Diet» Architecture
Alibaba proposed a radical exit from this crisis by presenting the Qwen3.8-Flash-Next architecture—the technological foundation for the future Qwen4 series. Engineers implemented a concept separating intelligence into an «active mind» and an «external reference book.»

The central element became N-gram Embedding (PLE) technology. A traditional neural network stores all linguistic knowledge within its weights, which must be constantly multiplied. PLE, however, offloads local language patterns (bigrams and trigrams) into a separate table of about 20 million records. This table physically resides in standard server RAM, bypassing the expensive video memory of accelerators.

This engineering solution changes the economics of AI itself:

Division of Labor: Instead of engaging the full power of matrix cores to predict simple word combinations, the model performs a fast lookup query to the table. Network capacity grows almost without increasing computational costs.
Hybrid Memory: The model learns to use cheap addressable RAM for static knowledge and expensive VRAM only for dynamic reasoning. Despite weighing hundreds of billions of parameters, only about 6 billion are actually calculated per token thanks to selective Mixture of Experts (MoE) activation.
Radical Savings: This approach reduced machine time requirements during training by approximately 9 times compared to the previous generation (Qwen3.7-Plus).

However, this elegance requires strict memory discipline. The massive n-gram table occupies significant disk space and resists heavy compression. Practice shows that if data is accidentally evicted from fast video memory to system memory via the PCIe bus, generation speed drops more than tenfold, turning a powerful accelerator into a slow processor.

Building Semiconductor Skyscrapers
While algorithmists optimized rack volume usage, materials scientists offered a similar idea for the crystal itself. Research from the University of Hong Kong paves the way for moving Moore’s Law into the third dimension.

Introducing 2D materials allows breaking free from flat topology constraints. Part of the logic gates, conductors, and memory cells moves to the upper layers of the chip and its backside. The logic here mirrors Alibaba’s software layer:

Foundation Liberation: Just as pattern tables liberate VRAM, offloading auxiliary circuits to upper levels relieves the overheated lower silicon layer where main computations occur.
Micro-scale Highways: Reducing inter-layer distances to micron ranges drastically cuts signal delays and current leakage. Energy is no longer spent pumping data through long copper traces on printed circuit boards.
Material Specialization: Different levels of a 3D chip can consist of different compounds. Some layers are optimized for high-speed data transfer, others for energy-efficient storage—something impossible to implement in a pure silicon monolith.

Architecture Over Brute Force
Both phenomena—the software offloading of data to RAM and the hardware erection of layers above the crystal—are symptoms of one deep transformation. The industry shifted from chasing peak FLOPS to fighting for the efficient use of available area and volume.

Alibaba proves mathematically that information processing costs decrease not by adding new GPUs, but by refusing unnecessary calculations. Mechanisms like Qwen Sparse Attention (QSA) work as intelligent context filters, forcing the system to analyze only relevant text fragments rather than the entire million-token sequence.

The semiconductor industry responds by building volumetric circuits where scarce horizontal area is compensated by vertical growth.

This symbiosis shapes the next decade’s landscape. Future supercomputers, such as clusters based on Zhenwu V9000, will physically be unable to efficiently process old monolithic models weighing trillions of parameters—they simply lack the memory bandwidth. But they will become an ideal environment for sparse architectures capable of distributing their mass between the chip’s fast memory and scalable arrays of server RAM. Artificial Intelligence ceases to be a heavyweight lifted by ever more hands, becoming a complex spatial structure where every unit of volume works at maximum output.

Asymmetric Activation Instead of Monolithic Load

Traditional systems operated on total mobilization principles: any task engaged all available transistors or neural network weights, leading to colossal idle energy losses. Modern engineering dictates a different logic—resources must connect asymmetrically, strictly when needed.

Dispatching Instead of Blind Computation
The architecture of future Qwen4 models is based on the Mixture of Experts (MoE) principle, transforming a giant neural network from a single inflexible canvas into a flexible constructor of specialized modules.

Instead of making the entire network think over every token, engineers introduced an intelligent routing mechanism. The gating network acts as a distribution center dispatcher: it instantly evaluates an incoming request and directs it only to those expert nodes most competent in the given topic. If a user asks about quantum physics, corresponding mathematical subnets activate; if a poem needs generating, language experts wake up.

Other modules remain in deep sleep, physically consuming no processor cycles. This creates a phenomenon of computational hyper-efficiency:

Weight-to-Load Gap: A model may contain conditional 10 trillion parameters (like future Qwen5), but use only a fraction of them—billions of active weights—to generate a single response. In the Qwen3.8-Flash-Next experiment, this gap is even clearer: out of roughly 180 billion total, only about 6 billion are actually calculated.
Reduced Cost of Ownership: It is precisely this sparse activation that makes inference of gigantic models economically viable. Electricity costs drop as the bulk of the silicon idles. This allows companies to scale to GigaChat-20B MoE or Llama 4 sizes without proportionally increasing data center expenses.

The price of this efficiency is training complexity. The dispatcher must perfectly balance load so some experts don’t overload while others atrophy from disuse. Complex penalty functions are applied to force tasks to distribute evenly across the academy of specialists.

Three-Dimensional Current Localization
Microelectronics solves the same energy-saving task but operates under Ohm’s law and thermodynamics. The transition from flat chips to 3D packaging marks the end of the era of long conductors.

In classical schemes, an electrical impulse must overcome several centimeters inside the crystal and tens of centimeters on the motherboard. Each such movement involves resistance and heat release. Three-dimensional integration via interposers and TSV transitions radically compresses space:

Distance Minimization: Memory layers (HBM) now sit literally atop compute layers. Signals no longer need to run a marathon along copper traces—they move vertically a microscopic height in nanoseconds.
Idle Time Elimination: Localizing computation within volume allows the system to operate differently. Like how software turns off unnecessary experts, hardware controllers can de-energize entire floors of a multi-layer crystal if current load is localized elsewhere. Energy stops dissipating across the board area and concentrates exactly where work happens.

Adaptive Intelligence
The convergence of these two approaches forms a new technological standard—dynamic infrastructure. The software layer learns to delegate tasks to specialized algorithms, while the hardware layer provides a physical environment where these signals are processed with minimal cost of traversing space.

For artificial intelligence, this means a qualitative leap. The system ceases to be a gluttonous furnace burning megawatts to maintain cluster temperature. It transforms into a highly organized structure operating on neurobiological economy principles: the maximum number of connections is stored potentially, but energy is spent only on synapses participating in the current act of thought. Computations become point-like, and architecture becomes alive.

From Benchmarks to System Throughput

Analysis of modern tech announcements reveals a tectonic shift in progress evaluation. Where venture capitalists and academia once looked at benchmark graphs (SWE-bench, MathVision), focus today has shifted to data center physics and silicon throughput. Success is measured not by model answer accuracy, but by a corporation’s ability to synchronize hundreds of thousands of compute nodes.

Mathematics of Scale
Alibaba taught the market a demonstrative lesson by announcing parameters for future Qwen4.5 and Qwen5 models (5–10 trillion weights) long before working prototypes or independent tests appeared. The exchange reacted with a 5.1% stock rise, evaluating not promised generation quality but demonstrated readiness to build a 500,000-accelerator Zhenwu V9000 cluster.

In this new reality, success metrics become physical infrastructure constants:

Interconnect as Foundation: For trillion-parameter models, critical resources are not individual core performance but communication speeds between them. T-Head chip bandwidth at 1200 GB/s (1.2 TB/s) is the circulatory system of future AI. If optimization gradients delay in the network even milliseconds, a half-million-node supercomputer becomes a pile of expensive metal.
Energy Over Algorithms: CEO Eddie Wu outlined a development horizon of 20 GW power by 2032. Today, AI development hits not transformer attention formulas but power plant capabilities and water cooling loop density. Progress is limited by kilowatt-hours.
Strategic Sovereignty: Alibaba leadership openly admits the Western dependency era ended. Full-stack AI creation became a survival question amid export controls. Parameter counts (those same 10 trillion) now serve merely as political statements of technological independence, while real advantage lies in controlling one’s own HBM supply chain.

Vertical Limit
Parallelly in microelectronics, there is abandonment of the industry’s sacred cow—the nanometer race. When elements cannot shrink further without violating quantum mechanics, engineers begin looking upward.

For 3D chips, volumetric characteristics became priority:

Packing Density: The main challenge is no longer gate length but the number of layers that can be fused into a single monolith via TSV transitions without destroying the crystal from thermal expansion.
Heat Hydrodynamics: In a multi-layer structure, heat is locked inside the «pie.» Victory goes to whoever’s microchannel architecture better sideways evacuates joules of energy through interposers, allowing stability under extreme loads.
Data Locality: As noted by T-Head Vice President Gao Hui, equipping a chip with 216 GB HBM memory minimizes data movement. In such a system scale, any transaction beyond the local node costs too much in time and energy.

Software-Hardware Integration
Modern artificial intelligence has outgrown software status. A 10-trillion-parameter model physically cannot exist outside an ecosystem designed for it. Final synthesis occurred: MoE routing algorithms must perfectly match physical interconnection network topology.

Victory in the next phase of technological confrontation won’t go to the author of the cleverest mathematical loss function. The triumphant will be the closed-loop control architect—one who can build a unified system where the software task scheduler understands thermal maps of each of the 500,000 floors of a semiconductor skyscraper. AI development finally left pure mathematics and entered heavy industry, megawatt logistics, and precision temperature control.

Conclusion

Artificial intelligence ceased being exclusively software product launchable on any suitable hardware. Today it represents an inextricable symbiosis of algorithm and architecture.

Sparse activation mechanisms (MoE) and external knowledge tables (N-gram Embedding) require instantaneous data movement between memory layers without delays. Conversely, 3D chips with localized computing and specialized layers become physically meaningless without software capable of distributing load point-wise, avoiding heating the entire crystal. Software’s «computational diet» found its perfect embodiment in semiconductor «skyscrapers.»

Artificial intelligence has definitively matured. It exited virtual pure mathematics space and transformed into a complex spatial structure subject to thermodynamics and macroeconomics laws. The next decade will belong to systems engineers capable of designing a unified control loop where the software task scheduler understands multilayer silicon thermal maps, and physical crystal architecture is ideally honed for neural networks’ asymmetric logic. The race for horizontal scale is over; the era of depth has begun.

Let's start a conversation

Start Scaling Today

Contact our experts to select the optimal hardware configuration and calculate rental pricing for your AI workloads.

Location: Moscow, ext.ter.g., sh. Ochakovskoe, 28, p.2
GPU Servers High-density compute
Облачная аренда Быстрый старт ИИ моделей
Enterprise SLA Guaranteed availability
Low Latency Optimal network routes