[ News ] Huawei Reportedly Aims 70% GPU Utilization with New AI ...

In late November 2025 Huawei unveiled an ambitious software stack that promises to lift the utilisation of its existing Ascend GPUs from roughly

30–40 %

to about

70 %

. The announcement, echoed by TrendForce and NotebookCheck, outlines a unified resource‑management platform that will orchestrate compute across Huawei’s own chips, NVIDIA GPUs, and third‑party accelerators—mirroring the Run:ai solution NVIDIA acquired in 2024. For engineers, system architects, and product managers evaluating AI infrastructure, this development is more than a marketing headline; it signals a strategic pivot toward software‑centric optimisation that could reshape cost structures, accelerate time‑to‑value, and redefine competitive dynamics in China’s rapidly maturing AI ecosystem.

• Target utilisation boost: Huawei projects a 70 % GPU utilisation rate, doubling current productivity for Ascend clusters.

• Unified orchestration: The new stack will manage Huawei Ascend chips, NVIDIA GPUs, and other accelerators under a single API, simplifying multi‑vendor deployments.

• Strategic alignment: The move dovetails with China’s semiconductor self‑reliance policy and offsets limited access to the latest Western GPUs.

• Business impact: Mid‑tier enterprises could adopt Huawei’s stack as a cost‑effective alternative to AWS/GCP, while large AI labs can extract more performance from existing hardware.

• Actionable takeaways: Evaluate integration effort, benchmark against current workloads, and consider hybrid cloud strategies that leverage the new orchestration layer.

The 70 % utilisation claim is not a marginal tweak; it represents a potential

double‑digit

shift in the economics of AI compute. In a market where GPU costs are still high and geopolitical constraints limit hardware options, any software layer that can squeeze more performance out of existing silicon offers a compelling value proposition.

Cost‑per‑Inference Reduction

• Assuming a typical inference workload consumes ~0.5 TFLOPs/s on an Ascend 950, doubling utilisation would halve the time per inference, directly cutting GPU hours and associated power consumption.

• For a mid‑tier enterprise running 10,000 inferences per day, this could translate to savings of several million RMB annually—well within the budgetary range for many AI startups and SMEs.

Accelerated Time‑to‑Market

• A higher utilisation rate means models can be trained or fine‑tuned faster. If a model that previously required 48 hours now completes in 24, product cycles shrink accordingly.

• Competitive advantage emerges for firms that need to iterate rapidly on generative AI, reinforcement learning, or large‑scale recommendation systems.

Hardware Portfolio Flexibility

• The orchestration layer’s ability to manage NVIDIA GPUs alongside Huawei chips provides a hybrid path. Enterprises can keep legacy NVIDIA workloads while gradually migrating to Ascend for cost savings.

• Software‑centric optimisation reduces the need for constant hardware refresh cycles, aligning with capital‑intensive data‑center investment strategies.

Deploying Huawei’s new orchestration stack involves several key steps that align with best practices in modern AI platform engineering. Below is a pragmatic roadmap tailored for system architects and DevOps teams.

• Hardware inventory: Catalogue all Ascend GPUs, NVIDIA GPUs, and any third‑party accelerators (e.g., Habana, Cerebras) in the cluster.

• Software stack audit: Verify current driver versions, firmware updates, and existing orchestration tools (Kubernetes, OpenShift).

• Workload profiling: Use tools like Huawei’s UCM or NVIDIA Nsight to capture baseline utilisation metrics for representative inference and training jobs.

• Deploy the orchestration agent: Install the Huawei software agent on each node, ensuring compatibility with existing container runtimes (Docker, CRI‑O).

• API integration: Expose the unified API endpoints to your CI/CD pipeline. The stack supports REST and gRPC interfaces compatible with Kubernetes Custom Resource Definitions (CRDs).

• Resource class definition: Define resource classes for Ascend, NVIDIA, and third‑party accelerators. This enables fine‑grained scheduling policies.

• Affinity & anti‑affinity rules: Configure rules to co‑locate complementary workloads (e.g., GPU‑heavy training with memory‑intensive inference) across heterogeneous devices.

• Priority classes: Assign higher priority to latency‑sensitive inference jobs, ensuring they receive immediate access to underutilised accelerators.

• Dynamic scaling: Leverage the stack’s auto‑scaling hooks to spin up additional nodes during peak periods without manual intervention.

• Benchmark suite: Run industry benchmarks such as MLPerf Inference v2.0, GPT‑4o fine‑tuning workloads, and custom models (e.g., DeepSeek) to capture utilisation improvements.

• Monitoring dashboards: Integrate with Prometheus and Grafana to visualize real‑time utilisation, queue lengths, and job completion times.

• A/B testing: Deploy a subset of jobs on the new orchestration layer while keeping others on legacy scheduling to quantify gains accurately.

• Access controls: Enforce role‑based access via RBAC, ensuring only authorised teams can submit high‑priority jobs.

• Audit logging: Capture job metadata, resource allocation decisions, and performance metrics for compliance and optimisation insights.

• Cost tracking: Use the stack’s built‑in cost‑allocation module to attribute GPU usage to business units or projects.

While Huawei’s Ascend GPUs have historically lagged behind NVIDIA’s latest architectures in raw FLOPs, the proposed software layer aims to close that gap through utilisation efficiency. The comparison below highlights key differentiators.

NVIDIA (Ampere/Blackwell)

Huawei Ascend (950PR/DT, 960, 970)

Peak FP16 Performance

~30 TFLOPs per GPU (A100)

~18 TFLOPs per chip (Ascend 970)

Memory Bandwidth

1550 GB/s (HBM2e)

1400 GB/s (HBM3)

Software Stack

CUDNN, TensorRT, Triton Inference Server

Ascend DL Framework, UCM, Unified Orchestration Layer

Utilisation Target

~40 % (industry average)

~70 % (Huawei claim)

Multi‑vendor Flexibility

Limited to NVIDIA ecosystem

Native support for NVIDIA + third‑party accelerators

The table illustrates that while raw performance remains a factor, Huawei’s strategy focuses on

efficiency and flexibility

. For enterprises already invested in Ascend hardware, the software upgrade could deliver ROI without new silicon purchases.

Estimating return on investment requires translating utilisation gains into tangible cost savings. The following simplified model assumes a data‑center cluster of 10 Ascend 970 GPUs, each costing ~¥3 million RMB (approx. $400k). GPU operating costs include electricity (~$0.02 per kWh) and maintenance.

Baseline scenario (35 % utilisation)

• Average inference throughput: 500 inferences/second

• Daily inference volume: 43 million

• Annual GPU hours: 10 GPUs × 24 h/day × 365 ≈ 87,600 GPU‑hrs

• Electricity cost (assuming 300 W per GPU): 0.3 kWh/h × 87,600 ≈ ¥26,280 RMB/year (~$4k)

Optimised scenario (70 % utilisation)

• Throughput doubles to 1,000 inferences/second

• Daily volume: 86 million

• GPU hours effectively halved for the same workload: ~43,800 GPU‑hrs/year

• Electricity cost halves to ~¥13,140 RMB (~$2k)

• Additional software licensing (assumed ¥200k): negligible compared to savings.

Net annual saving:

Approximately ¥13,000 RMB ($2k) per cluster. While modest in absolute terms, this figure scales with cluster size and workload intensity. For a 100‑GPU deployment, the savings jump to ~¥1.3 million RMB (~$200k), making the upgrade financially attractive.

Adopting Huawei’s orchestration layer is not without risks. Below are common obstacles and practical solutions.

• Compatibility with Existing Workflows: Legacy pipelines may rely on NVIDIA‑specific APIs. Mitigation: Use containerised adapters that translate between Huawei DL Framework calls and standard CUDA kernels where possible.

• Vendor Lock‑In Concerns: While the stack supports multiple accelerators, deep integration with Huawei’s ecosystem could create dependency. Mitigation: Maintain a hybrid cluster strategy, gradually migrating workloads while keeping fallback NVIDIA nodes.

• Skill Gap: Engineers accustomed to CUDA may need training on Ascend programming models. Mitigation: Leverage Huawei’s official training modules and community forums; consider hiring consultants for initial ramp‑up.

• Performance Variability Across Workloads: The 70 % utilisation claim is aggregate; specific models (e.g., transformer inference) may not see proportional gains. Mitigation: Conduct workload‑specific benchmarks before full deployment.

The convergence of software orchestration and heterogeneous silicon heralds a new era in AI infrastructure:

• Software‑Defined AI Fabric: Future stacks will abstract compute, memory, and networking into programmable fabrics, enabling instant reconfiguration for emerging workloads (e.g., multimodal LLMs).

• Edge‑to‑Cloud Continuity: Huawei’s unified API could extend to its 5G edge platforms, allowing seamless migration of models from on‑prem data centers to mobile gateways.

• AI‑Optimised Network Topologies: Coupling the orchestration layer with Huawei’s CloudMatrix 384 architecture may unlock sub‑millisecond inter‑node communication, essential for real‑time inference at scale.

• Open Ecosystem Momentum: By supporting open models like DeepSeek and integrating with community frameworks (e.g., PyTorch, TensorFlow), Huawei positions itself as a credible partner for AI research labs worldwide.

• Conduct an Internal Utilisation Audit: Measure current GPU utilisation across all accelerators to establish a baseline and identify bottlenecks that the new stack could address.

• Prototype on a Small Cluster: Deploy the orchestration layer on a subset of nodes, run representative workloads (GPT‑4o fine‑tuning, image classification), and capture utilisation metrics.

• Engage with Huawei Technical Support: Request detailed integration guides, API documentation, and case studies from enterprises that have piloted the stack.

• Assess Hybrid Cloud Strategy: Evaluate whether a mix of Huawei Ascend clusters and NVIDIA GPUs in public clouds (e.g., Tencent Cloud) could deliver optimal cost‑performance balance.

• Build a Cost‑Benefit Model: Incorporate projected utilisation gains, energy savings, licensing costs, and potential revenue uplift from faster model deployment into your ROI calculator.

• Plan for Skill Development: Allocate budget for training sessions on Ascend DL Frameworks and the new orchestration APIs to ensure smooth adoption.

Huawei’s 2025 AI‑infrastructure push represents a bold attempt to shift the value equation from silicon to software. For enterprises that already own Ascend hardware—or are considering it—this development could deliver immediate, tangible benefits while positioning them for future scalability. The next few months will be critical: independent benchmarks and real‑world deployments will confirm whether the promised 70 % utilisation translates into a sustainable competitive edge.