On Monday, Amazon Web Services released a tool called TorchNeuron that allows AI startups to easily move their AI applications onto Amazon’s AI chip, Trainium. In doing so, Amazon is also directly attacking the software advantage that has helped Nvidia become the default hardware choice for AI applications and spurred its rise as the AI hardware provider of choice.
TorchNeuron works with PyTorch, a free and open source toolkit that developers use to build, train, and run AI models. According to Amazon, it allows AI applications written with PyTorch to run on Trainium by changing a single line of configuration – no code rewriting required.
Amazon claims that around 85% of AI applications are written with PyTorch. Facebook initially developed it, but since 2022 it has been part of the Linux Foundation, which maintains open source projects. Nvidia has spent years ensuring that its CUDA software – the software infrastructure that allows applications to run on Nvidia GPUs and its other hardware – seamlessly supports AI applications written with PyTorch. As a result, Nvidia’s hardware is now deeply integrated into the toolbox, essentially by default.
TorchNeuron is adapted from PyTorch, so developers who are familiar with PyTorch already know how to use it. Amazon is not the first company to try to easily introduce its chips onto the platform. PyTorch users can also port their applications to AMD GPUs or even Apple chips with a simple configuration change.
However, Amazon claims that porting to Trainium isn’t just easy; it also gives AI startups access to Amazon’s entire cloud. Unsurprisingly, AWS says this is a better choice for startups than using neoclouds, many of which are tied to Nvidia and its hardware, like Coreweave, Nebius, and Firmus.
“You build on AWS. You don’t bet on a single-purpose compute vendor and go out of your way to build everything else around it,” Colin Brace, vice president of technology software at AWS Annapurna Labs (the unit that makes Amazon’s custom chips) told TechCrunch via email. “On AWS,” he continued, “the infrastructure needed to go from prototype to production is already there, allowing a founder to scale quickly and scale when they need to.” »
AWS claims that because the same chip handles both training (teaching the AI model) and inference (operating the finished model, such as when answering a user’s question), and is tightly integrated with the cloud infrastructure that startups need, its cloud will save them money, even if AWS doesn’t offer the cheapest price per token.
“The value of Trainium lies in its value for money and that it is a single platform for training and inference, which improves cost effectiveness across a full workload,” Brace said. “Look at Deepgram: they migrated to Trainium using native PyTorch support and they process 70% more audio per dollar with production-grade accuracy.” Deepgram is an AI voice application provider, competing with ElevenLabs.
But Amazon isn’t just giving PyTorch app developers an easy option to migrate to its cloud.
Amazon also says it’s pushing PyTorch to be less automatically Nvidia-centric, and that it will provide the code to do so to the main PyTorch project. In the blog post announcing the new tool, the company wrote:
“Amazon’s approach extends beyond its own hardware. The company is making upstream changes to PyTorch itself, working with the PyTorch community to remove code that ties the framework to specific chips. The goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users.”
While such a tactic is undoubtedly good for those selling hardware and cloud services competing with Nvidia, one of PyTorch’s goals is to hide all the pesky difficulties involved in running AI code on hardware. We’ll have to wait and see if pushing Nvidia this far gives developers more choice and more affordable options, as Amazon promises, or just more configuration hassles.
Either way, if Amazon manages to deeply decouple PyTorch from Nvidia, it could have far bigger ramifications for the GPU giant than simply dealing with chip competitors. As Brace said: “The real story is removing the lock-in that prevented developers from having real choice. »
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
Gn bussni
