Meta Goes All-In on AMD's MI300.
Update
Edited by Brian Birnbaum and an update of my original AMD deep dive.
AMD has an advantage over Nvidia in the inference market, which will be much larger than the training market. This is best evidenced by Meta’s big bet on AMD’s MI300X chip.
Analysts are increasingly unsure about AMD, with HSBC’s Frank Lee double downgrading the stock last week from buy to sell. Meanwhile, Meta’s Mark Zuckerberg has gone all in on AMD by running Llama’s inferences on AMD’s MI300X exclusively. Zuckerberg’s move announces loud and clear what the market is failing to grasp: that AMD has achieved inference superiority; and that the inference market is going to be much larger than the training market.
Once you train an AI model, you make many inferences. The demand for training GPUs is behind Nvidia’s meteoric revenue growth, as depicted below. So if AMD is as well positioned for the inference market as I believe, we are likely to see a similar revenue growth curve in the years ahead.
My view of AMD’s positioning for the inference market stems from a first-principles understanding of the physics. A chip’s function is to entice electron movement in such an accurate fashion as to perform complex arithmetic operations. Since AI models are quite large, the closer you can place the memory engine to the actual compute engine, the less distance the electrons must travel. This equates to lower latency and ultimately faster and cheaper inferences.
ADM’s inference advantage lies in its chiplet platform, which allows the company to mix and match different compute engines at a marginal cost. The MI300X platform has enabled them to introduce more memory on-chip than Nvidia’s alternative, which is ultimately why Meta has picked the MI300X to make inferences with Llama.
Inthe below interviewfrom a year ago (snippet starts at 12:38), Lisa explains that AMD has oriented the MI300 to capitalize on the massive inference opportunity ahead. She also explains that there won’t be a single chip for all AI workloads but rather a broad range that will excel at specific AI workloads. Thus, while legacy analysts are looking for signs of traction on the training side, they are completely missing the point.
Meta going all in on the MI300X is testament to the power of AMD’s platform. AMD’s chiplet platform enables them to repurpose the chip to any specific AI workload at a marginal cost. As the demand for specialized workloads arises, we’ll see AMD pull ahead–as we are seeing with inference now. Indeed, legacy analysts are not only failing to understand AMD’s edge on the inference side, but also the long-term implications of its platform.
As evidence of this, consider the difference between the MI300A and the MI300X. The MI300A contains three CPU tiles where the MI300X contains three GPU tiles, ultimately catering to decidedly disparate AI workloads. The cost of this modification is marginal for AMD because the chiplet platform allows them to swap computing units within the chip easily.
Folks on X are finding it difficult to believe that Meta’s Llama runs inferences exclusively on the MI300X. I did them the favor of uploading a snippet of AMD’s Advancing AI event in October 2024 (snippet starts at 50:38). At 1:11 you will hear Meta’s Kevin Salvatori, VP of Infrastructure Supply Chain and Engineering, say that “all Meta live traffic has been served using MI300X exclusively due to its large memory capacity and TCO (total cost of ownership) advantage.” By live traffic he is referring to inferences.
Following this statement are three critical datapoints:
Meta is working on deploying MI300X for training workloads too (1:33).
Both Meta and AMD are culturally aligned around the idea of open software, versus Nvidia’s closed ecosystem (1:37).
The feedback loop between the two companies is fast across the full stack, with them collaborating on the MI350 and MI400 series already (2:45).
In the coming years, AMD is likely to have an “Nvidia moment” as the inference market grows. In my Mongo DB deep dive I learned that AI apps have not achieved widespread traction yet. But I believe Meta’s evolution is a taste of things to come. Their apps have become incredibly addictive over the past two years, essentially driven by AI inferences. I don’t think it will take long for the rest of the economy to become inference-driven, greatly benefiting AMD.
The conversation between Lisa Su and Meta’s Kevin Salvatori reveals that the two companies are working together for the long term. An underappreciated characteristic of AMD’s chiplet platform is that it enables rapid iteration. AMD dethroned Intel by working closely with customers and using their feedback to make better products. The same dynamic is at play, with a market that’s set to grow explosively.
Until next time!
⚡ If you enjoyed the post, please feel free to share with friends, drop a like and leave me a comment.
You can also reach me at:
Twitter: @alc2022
LinkedIn: antoniolinaresc
Thanks Antonio,your analysis is really helpful and easy to understand!
Thanks Antonio. AMD 2024Q4 annual report has been released, could you please share your opion.