The server market is about to experience a major shift, and it’s all thanks to GPUs. Omdia, a market researcher, predicts that GPUs are going to dominate the market in the near future. In fact, during the second quarter of this year alone, over 900 tons of Nvidia H100 GPUs were shipped. That’s a staggering amount!
This revelation comes from Omdia’s Cloud and Data Center Market Update, which highlights the ongoing change in datacenter investment priorities. The report explains that the demand for servers equipped with eight GPUs for AI processing has caused average prices to rise. At the same time, it has put pressure on investment in other areas.
As a result, Omdia has revised its estimate for annual server shipments in 2023, reducing it by another 1 million units to 11.6 million. This represents a 17 percent decline compared to last year. However, despite the decrease in unit shipments, the average price of a server has increased by more than 30 percent quarter-on-quarter and year-on-year. This is because hyperscalers are pouring their investments into these high-spec GPUs.
What’s interesting is that each Nvidia H100 GPU weighs around 3kg, and over 300,000 of them have been used by server makers. This adds up to the mind-boggling weight of 900 tons! Can you believe it?
It’s clear that the massive demand for AI servers with GPUs is primarily driven by hyperscale companies and cloud providers. These entities play a crucial role in the server market, benefiting the white box vendors, also known as the original design manufacturer (ODM) server makers.
Evidence suggests that ZT Systems, a US-based cloud server manufacturer, is behind a significant portion of the eight-GPU servers shipped in the second quarter. Apparently, as much as 17 percent of Nvidia’s $13.5 billion second-quarter revenue came from this single vendor. Even Nvidia’s CEO, Jensen Huang, showcased a DGX server manufactured by ZT Systems during Computex Taipei.
Omdia also believes that Meta is a major player contributing to the growth in demand for eight-GPU servers. Nvidia has previously stated that a single cloud service provider (likely referring to Meta) accounted for 22 percent of its revenue in the second quarter.
The report predicts that the trend of adopting servers optimized for AI processing will continue in the second half of this year and the first half of 2024. However, Omdia expects a better-than-seasonal rebound in demand for general-purpose servers in the current and next quarters due to increasing enterprise demand. This is anticipated to drive revenue growth of 13 percent in the third quarter and 29 percent in the fourth quarter, compared to the previous year.
Interestingly, Omdia forecasts that the continued deployment of eight-GPU servers will result in a 51 percent year-over-year growth in server market revenue during the first half of 2024. They even predict that one million H100 GPUs will find their way into systems!
But while there seems to be rapid investment in AI training capabilities among hyperscalers and cloud providers, this shouldn’t be mistaken for rapid adoption. According to a survey, only 18 percent of US adults have used ChatGPT, showing that AI usage is still relatively low. Additionally, generative AI represents only a small fraction of cloud computing costs for enterprises.
It’s also important to consider the downsides of relying on power-hungry GPU systems for AI training. One industry executive warned that if we don’t take action to implement more sustainable practices, AI will “burn the world.” Speaking at the International Broadcasting Convention, Tom Dunning, CEO and founder of Ad Signal, emphasized the urgent need to minimize the carbon impact of AI. He stated that data centers are already responsible for a significant portion of carbon dioxide emissions and that AI’s adoption is projected to grow rapidly.
In conclusion, the server market is undergoing a significant transformation with the increasing dominance of GPUs. While this presents exciting opportunities for AI processing, it also raises concerns about sustainability and carbon emissions. It will be fascinating to see how the market evolves in the coming years and how companies address these challenges.