AI Infrastructure 2026: New GPU, Optical Network and Immersive Cooling Requirements

Why AI infrastructure will look very different by 2026

AI infrastructure is entering a new phase. By 2026, next‑generation GPUs, optical interconnects and immersive cooling will reshape how data centers are designed, powered and operated.

The acceleration of generative AI and large language models is driving unprecedented compute density. High‑end accelerators such as NVIDIA H100 class GPUs already reach up to 700 W of thermal design power (TDP) per device, with 4–8 GPUs per server common in AI nodes.nvidia.com At rack scale, new architectures announced for 2026 can host 70+ GPUs and deliver several exaFLOPS of AI performance per rack, dramatically increasing local power and cooling demands.tomshardware.com

In parallel, a U.S. Department of Energy–backed study projects that national data center power use could nearly triple by 2028, reaching 6.7–12% of total U.S. electricity consumption, with AI as a primary driver.reuters.com Traditional air‑cooled designs and copper‑based networks struggle to follow this curve.

For Score Group, which integrates energy, digital infrastructure and new technologies, this shift is central to how we help organisations prepare. Our divisions Noor Energy, Noor ITS and Noor Technology work together to align AI performance ambitions with power, cooling and networking realities—so that innovation does not outpace the data center that must host it.

1. GPU and accelerator requirements in 2026‑class AI clusters

From standalone servers to exaFLOP GPU fabrics

AI compute is moving from isolated GPU servers to tightly coupled accelerator fabrics:

• High‑power GPUs as standard: H100‑class accelerators deliver multi‑petaFLOP throughput per node with TDPs up to 700 W per GPU.nvidia.com

• Large GPU counts per rack: New 2026 platforms from leading OEMs scale to ~72 GPUs per rack with several terabytes of high‑bandwidth memory, targeting ~2–3 exaFLOPS of low‑precision AI performance per rack.tomshardware.com

• GPU fabrics, not just buses: Technologies like NVLink, proprietary accelerator links and high‑performance Ethernet/InfiniBand fabrics create a logical “super‑GPU” across racks.

These trends mean that “AI infrastructure 2026” is less about one powerful server and more about the cluster as a system: compute, memory, storage, network and cooling must be designed together.

Density, power and cooling implications

Industry surveys show that typical data center racks today still average below 10 kW, with few facilities operating racks beyond 30 kW.intelligence.uptimeinstitute.com Yet AI‑optimized racks in hyperscale environments already approach or exceed 80–100 kW per rack.techradar.com This creates a clear tension between existing facilities and upcoming AI requirements.

A practical way to think about 2026 power and cooling envelopes is:

• 10–30 kW/rack: High‑efficiency air cooling with strong containment and possibly rear‑door heat exchangers.

• 30–80 kW/rack: Direct‑to‑chip liquid cooling becomes primary, often with warm‑water loops.

• 80–150 kW+/rack: Direct liquid plus immersion or other advanced solutions for AI‑dense racks.

Standards bodies such as ASHRAE already highlight liquid and immersion approaches for highly dense datacom equipment, noting that some components will require liquid to stay within thermal envelopes.handbook.ashrae.org

What this means for your organisation by 2026

For most enterprises and service providers, the key questions are not only “which GPU?” but:

• Can my existing rooms host even a small AI pod at 30–60 kW per rack?

• Do my electrical distribution and UPS systems support these step changes?

• Can my cooling plant and building systems absorb new heat loads efficiently?

At Score Group, we treat this as a cross‑disciplinary problem:

• Noor ITS designs and optimises the digital infrastructure—racks, power trains, data center layout, high‑speed fabrics and resilience (PRA/PCA).

• Noor Energy addresses grid connection, on‑site generation, storage, building technical management and smart energy monitoring.

• Noor Technology focuses on AI workloads themselves, automation (RPA) and smart connectivity, so the infrastructure matches real computational needs.

This tripartite approach helps avoid over‑ or under‑engineering AI infrastructure as you plan toward 2026.

2. Optical networking: preparing for a photonic backbone

Why copper and pluggable optics are hitting a wall

AI clusters stress networks differently from traditional enterprise workloads. Massive east‑west traffic between GPUs demands ultra‑high‑bandwidth, low‑latency fabrics. As speeds reach 800 Gb/s and beyond, conventional copper and even many pluggable optical modules become inefficient or impractical at rack‑scale distances.

Leading vendors have shown that traditional pluggable optics can consume around 30 W per 800G port, while emerging co‑packaged optics can reduce this to roughly 9 W per port and dramatically cut signal loss by integrating the optical engine directly next to the switch ASIC.tomshardware.com At AI cluster scale, this translates into significant savings in power, heat and rack space.

Silicon photonics and co‑packaged optics by 2026

The industry is converging on silicon photonics and co‑packaged optics (CPO) as foundational for next‑generation AI data centers:

• NVIDIA has presented a roadmap where its 2026 networking platforms (InfiniBand and Ethernet) adopt silicon photonics and CPO to support ultra‑high‑speed links, treating the optical shift as “essential” rather than optional.tomshardware.com

• TSMC’s COUPE platform enables compact photonic engines; recent demos have shown up to 100 Tb/s of optical bandwidth per accelerator using silicon photonics chiplets.tomshardware.com

• Chip vendors such as AMD and networking specialists like Marvell are acquiring photonics companies to strengthen co‑packaged optics and photonic fabrics for AI systems, underscoring how strategic these technologies have become.reuters.com

By 2026, facilities that deploy leading‑edge AI clusters will likely rely on some combination of high‑speed optical fabrics, internal photonic links and carefully engineered fiber infrastructure.

Network architecture impacts

Preparing for this optical future does not mean every site must rip and replace today. It does mean that 2025–2026 designs should:

• Adopt leaf‑spine or more advanced Clos architectures that can evolve to 400/800G and beyond.

• Ensure fiber‑rich plant and patching, with pathways sized for future high‑count cables and MPO/MTP connectivity.

• Plan for AI‑specific fabrics (e.g. modern InfiniBand or lossless Ethernet with RDMA) separate from general IT traffic.

• Integrate network telemetry, timing and out‑of‑band control from day one for observability.

Our Noor ITS teams typically address these aspects in tandem with power and cooling design, so that GPU fabrics, optical links and energy systems evolve together instead of in silos.

Key AI infrastructure requirements by 2026: a comparative view

Domain	2024 Baseline	2026 AI‑ready Target	Impact on Design
Compute / GPUs	4–8 GPUs per node, limited GPU fabrics	Dozens of GPUs per rack, exaFLOP‑class pods	Higher rack power, tighter thermal budgets, advanced scheduling
Rack Density	<10 kW/rack on average, few >30 kWintelligence.uptimeinstitute.com	30–80 kW common for AI pods; >80 kW for dense training rackstechradar.com	Liquid cooling and possible immersion; reinforced power distribution
Networking	100–400G ports, copper plus pluggable optics	400–800G+ optical with emerging co‑packaged opticstomshardware.com	Fiber‑centric designs, new switch platforms, optical management
Cooling	Room‑level air; limited liquid adoption	Direct‑to‑chip liquid and growing immersion cooling penetrationprecedenceresearch.com	Facility water loops, fluid management, new O&M processes
Energy / Sustainability	PUE ~1.5, data center power ~4% of U.S. electricityreuters.com	Higher absolute power, pressure for greener and more efficient designs	On‑site generation, renewables, waste‑heat reuse, advanced EMS

3. Immersive and liquid cooling: from niche to necessity

Why air cooling alone is no longer enough

As GPU heat flux rises, traditional air‑only cooling reaches practical limits. Even with aggressive containment and rear‑door heat exchangers, moving enough air through 50–100 kW racks is challenging, noisy and often inefficient.

The market response is clear: multiple independent analyses estimate that the global data center immersion cooling market, valued at around USD 1.5–1.8 billion in 2024–2025, could grow at roughly 18–24% CAGR to reach the multi‑billion‑dollar range by the early 2030s.precedenceresearch.com This growth is closely tied to AI and high‑performance computing workloads.

Immersion and liquid cooling are attractive because they can significantly reduce cooling energy consumption—some studies report up to ~50% savings versus comparable air‑cooled deployments—and support much higher rack densities.grandviewresearch.com

Immersion cooling options and standards

In practice, you will encounter three main categories:

• Enhanced air cooling: Hot/cold aisle containment, rear‑door heat exchangers, in‑row coolers. Effective up to ~30–40 kW per rack in many cases.

• Direct‑to‑chip (D2C) liquid cooling: Cold plates attached to CPUs/GPUs with facility water loops. Supported by ASHRAE guidance and increasingly by server OEMs.handbook.ashrae.org

• Immersion cooling: Entire servers submerged in dielectric fluid (single‑phase or two‑phase). ASHRAE recognises these approaches, noting that immersion is a viable option for highly dense datacom equipment, using engineered liquids such as mineral oils or fluoroketones.handbook.ashrae.org

Immersion cooling is no longer an exotic experiment; it is emerging as a mainstream answer to high‑density AI racks, with North America currently the largest regional market.precedenceresearch.com

How immersive cooling reshapes facility design

Adopting immersion or advanced liquid cooling affects much more than the rack:

• Mechanical plant: New heat exchangers, pumps and fluid distribution need to integrate with existing chillers or dry coolers.

• Building design: Floor loading, service clearances and access routes may change due to immersion tanks and manifolds.

• Operations & maintenance: Handling of dielectric fluids, new safety procedures and training for technicians.

• Monitoring: Temperature, flow and leak detection sensors must feed into a modern BMS/EMS for real‑time optimisation.

These aspects are naturally cross‑functional, which is why at Score Group, Noor Energy focuses on the energy and building side (GTB/GTC, heat rejection, renewables), while Noor ITS addresses data center layout, equipment qualification and lifecycle operations. Noor Technology can then help exploit the resulting capabilities with AI‑driven monitoring and automation.

4. Energy, sustainability and regulatory pressures

The power crunch of AI data centers

The energy footprint of AI infrastructure is attracting global attention. The DOE‑backed study mentioned earlier suggests U.S. data center electricity use could rise from just over 4% of national consumption today to as high as 12% by 2028 if current AI trajectories continue.reuters.com

At the same time, Uptime Institute’s Global Data Center Survey shows average PUE values have remained nearly flat for several years, around 1.5, with only modest improvements in newer, larger sites.businesswire.com In other words: total energy is increasing faster than efficiency gains, especially as AI workloads roll out.

Regulators and clients are responding with stricter sustainability expectations, including reporting on carbon intensity, renewable sourcing and water use. Operators that cannot reconcile AI growth with ESG targets risk project delays or reputational damage.

Designing for lower PUE and carbon in an AI era

By 2026, an AI‑ready energy strategy typically includes:

• High‑efficiency cooling: Advanced air containment, free‑cooling where climate allows, liquid cooling and immersion where justified.

• Smart energy management: Fine‑grained metering, AI‑assisted analytics and automated setpoint optimisation across IT, cooling and building systems.

• Cleaner power mix: On‑site solar, storage, and contracts for low‑carbon electricity where grid capacity allows.

• Waste‑heat reuse: Capturing heat from high‑density AI clusters for building or district heating, a trend already visible in some European HPC sites.tomshardware.com

Here, the integrated positioning of Score Group—bridging energy, digital and new tech—helps align AI roadmaps with both operational efficiency and sustainability objectives.

5. A practical roadmap to 2026‑ready AI infrastructure

Step 1 – Clarify AI use cases and sourcing strategy

Not every organisation needs a hyperscale‑class cluster. Before investing in infrastructure, clarify:

• Which AI workloads are strategic (e.g. internal copilots, predictive maintenance, computer vision)?

• Which must run on‑premises for data sovereignty or latency reasons?

• Which can leverage public cloud or specialised AI hosting?

Noor Technology can support this early stage by evaluating AI and automation opportunities and identifying where on‑prem infrastructure truly adds value compared to cloud or hybrid strategies.

Step 2 – Assess power, cooling and network gaps

Next, compare target AI workloads with current facilities:

• Electrical capacity: Maximum kW per rack and per room, redundancy level, short‑circuit constraints.

• Cooling capability: Current kW/rack achievable with air, liquid or hybrid; potential to retrofit liquid loops or immersion zones.

• Networking: Existing fabrics (1/10/25/100G), fiber availability, scope for 400/800G and AI‑specific fabrics.

Our Noor ITS and Noor Energy teams typically conduct this as a combined audit, using both digital simulations and field measurements to define realistic upgrade paths rather than theoretical capabilities.

Step 3 – Pilot AI pods and iterate

Instead of a single “big bang” move, most organisations benefit from deploying a small AI pod first—perhaps a few racks with 30–60 kW density—then iterating:

• Deploy a limited GPU cluster with well‑defined AI use cases.

• Experiment with direct‑to‑chip or immersion cooling options where justified.

• Instrument the pod heavily—power, temperature, network, application metrics—and feed this data into an observability stack.

• Use insights to refine both infrastructure and workload placement before scaling.

This approach aligns with Score Group’s mission: combining practical digital infrastructure, smart energy solutions and innovative technologies in controlled, value‑driven steps.

FAQ: Key questions about AI infrastructure requirements for 2026

What rack power density should I plan for in a 2026 AI cluster?

Industry surveys indicate that typical racks today still average under 10 kW, but AI clusters already push 30–80 kW per rack, and some hyperscale deployments exceed 100 kW.intelligence.uptimeinstitute.com For most enterprise‑scale AI pods, planning for 30–60 kW per rack is a realistic starting point through 2026. This usually requires high‑efficiency air plus liquid cooling (direct‑to‑chip) and reinforced power distribution. If you foresee very dense training workloads or space constraints, consider designing at least one zone capable of 80–100 kW per rack using advanced liquid or immersion cooling.

Is immersion cooling mandatory for future AI workloads?

Immersion cooling is not mandatory for all AI workloads, but it becomes highly attractive at very high densities. Many inference‑focused deployments can operate effectively with enhanced air or direct‑to‑chip liquid cooling below ~40 kW per rack. However, for dense training clusters or where space is constrained, immersion can unlock 80–150 kW+ per rack and significantly reduce cooling energy, with market analyses showing strong uptake driven by AI and HPC.precedenceresearch.com The right answer depends on your workloads, energy costs, sustainability targets and facility constraints.

How soon do I need optical networking or co‑packaged optics?

Most organisations do not need to deploy co‑packaged optics immediately, but designs started in 2025–2026 should anticipate their arrival. Vendors are positioning silicon photonics and co‑packaged optics as essential for next‑generation AI fabrics launching around 2026, due to better energy efficiency and signal integrity at 800 Gb/s and beyond.tomshardware.com For new builds, that means ensuring fiber‑rich cabling, adequate space and power for future switch generations, and network architectures that can scale bandwidth without a full redesign.

Can existing enterprise data centers support 2026‑generation GPUs?

Many existing enterprise sites can host small numbers of 2026‑class GPUs, but often with constraints. Typical rooms were designed for 5–10 kW racks, while AI racks can require several times that.intelligence.uptimeinstitute.com Without upgrades, you may need to deploy GPUs sparsely or in dedicated aisles to avoid hot spots and overloading power trains. A focused assessment of electrical capacity, cooling plant, rack distribution and network links is essential. Frequently, the most efficient path is to create a dedicated AI pod—separate but integrated—rather than trying to retrofit the entire facility at once.

How does Score Group help organisations prepare for AI infrastructure 2026?

At Score Group, we approach AI infrastructure as a convergence of energy, digital and new technologies. Noor ITS designs and modernises network and data center infrastructures, including security and resiliency. Noor Energy delivers intelligent energy management, building control and renewable integration to keep AI growth compatible with sustainability goals. Noor Technology focuses on AI use cases, automation and smart connectivity so that infrastructure investments are driven by real business outcomes. Together, these divisions provide tailored, end‑to‑end support—from strategic roadmapping to integration and continuous optimisation.

What’s next?

AI infrastructure in 2026 will demand more from your GPUs, networks and cooling systems—but also from your energy strategy and digital operations. Rather than treating these as separate projects, consider them parts of a single transformation. If you are planning new AI workloads or rethinking your data center roadmap, the teams at Score Group can help you assess your current posture, define realistic next steps and design AI‑ready infrastructure where efficiency truly embraces innovation.