This post first appeared in the Constellation Insight newsletter, which features bespoke content weekly.
What's good for Nvidia shareholders may give enterprise technology buyers pause. Nvidia needs competition.
Following Nvidia's blowout second quarter results, surging margins and crazy demand it's clear that the company has little competition and a lot of pricing power. Nvidia's second quarter gross margin was 70.1%, up 43.5% from a year ago. Third quarter gross margins will creep up to 71.5% to 72.5%.
The competition won't begin to show up until the fourth quarter when AMD launches its AMD Instinct MI300A and MI300X GPUs and AWS re:Invent likely features new Trainium and Inferentia chips.
Sure, there are a few enterprises buying Nvidia systems to train models on-premises, but most will buy computing power through the cloud. Either way, Nvidia is getting paid enough that it will double its fourth quarter revenue in two quarters, assuming it hits its third quarter sales guidance of $16 billion.
Here are the moving parts enterprises need to consider as they explore and scale generative AI efforts.
- Model training isn't cheap and requires GPUs. Nvidia has pricing power on the infrastructure side.
- Today, that pricing power is absorbed because hyperscalers are building out and consumer Internet and enterprises see productivity returns. More than 50% of Nvidia’s data center revenue in the second quarter derived from cloud service providers.
- First movers will pay up for competitive advantage.
- Nvidia has worked on its supply chain network for a decade or more and now appears to be able to meet demand.
- Enterprises--especially those building their own infrastructure--will need to determine whether you go with Nvidia or consider options that'll be available in a few months.
To hear Nvidia CEO Jensen Huang tell it, there's only one choice. Not surprisingly, that choice is Nvidia. "The world has something along the lines of about $1 trillion worth of data centers installed, in the cloud, in enterprise and otherwise. And that $1 trillion of data centers is in the process of transitioning into accelerated computing and generative AI. We're seeing two simultaneous platform shifts at the same time," said Jensen.
Jensen continued:
"What makes NVIDIA special are: one, architecture. NVIDIA accelerates everything from data processing, training, inference, every AI model, real-time speech to computer vision, and giant recommenders to vector databases. The performance and versatility of our architecture translates to the lowest data center TCO and best energy efficiency."
It's hard to argue with Jensen now.
Going forward, the question will be whether everyone will need to train models with Nvidia or will something else do. Amazon CEO Andy Jassy will play the Nvidia game, but also noted there will be workloads for its own processors.
"Customers are excited by Amazon EC2 P5 instances powered by NVIDIA H100 GPUs to train large models and develop generative AI applications. However, to date, there's only been one viable option in the market for everybody and supply has been scarce,” said Jassy on Amazon's earnings conference call. “We're optimistic that a lot of large language model training and inference will be run on AWS' Trainium and Inferentia chips in the future."
You can also expect AMD to get training workloads too. AMD's value to the industry in x86 processors was being a counterweight to Intel. It will play the same role in GPUs and be formidable. AMD makes its case for generative AI workloads vs. Nvidia
AMD Instinct MI300A and MI300X GPUs are sampling to HPC, Cloud and AI customers with production in the fourth quarter. "I think there will be multiple winners. And we will be first to say that there are multiple winners," said AMD CEO Lisa Su.
