AMAZON

Company Announces Third-Generation Arm Processors (Graviton3)

AWS, the cloud arm of Amazon, announced EC2 C7g instances with Graviton3 technology, its third-generation Arm processors, and Trn1 instances that use Trainium AI accelerators, developed by AWS specifically for machine learning.

However, both announcements are preliminary. Therefore, for Graviton3, neither the generation of the Arm architecture, nor the number of cores, nor the frequency are specified. But it is argued that the new items compared to Graviton2-based instances perform a quarter better and twice as fast in floating point calculations and when working with cryptography. At the same time, they consume 60% less energy.

Some architectural changes are also mentioned. So support for bfloat16 allows you to speed up running AI tasks three times (we're probably talking about inference), and transitioning to DDR5 will increase memory bandwidth by 50%. Each vCPU has a dedicated cache and additional security is provided by hardware stack protection. Additionally, new instances use forced memory encryption by default, support for encrypted EBS volumes, and EFA connection speeds of up to 30 Gbps.

EC2 C7g is, according to AWS, suitable for HPC, EDA, analytics, gaming and advertising platforms, media encoding and more. Distributions of Amazon Linux 2, RHEL, SUSE and Ubuntu are available for them, as well as a number of already adapted software. Until now, the C7g has been available through closed trials upon request, but Epic Games, Formula 1 Management, Honeycomb.io and Twitter have all evaluated them.

The Graviton3 processors themselves, like their predecessors, will likely not be sold “outside” and outside the AWS core infrastructure will only enter Outposts. However, for Amazon itself, this – along with the Nitro implementation – is an important step towards independence from third-party providers and improving the performance of its own services. There are only 12 instance types currently offered based on Graviton2, but AWS is quite active and successful in migrating all other services and services to their own CPUs.

EC2 Trn1 instances generally aim to achieve the same goals. They are based on Trainium Accelerators, Amazon's second proprietary chip designed specifically for machine learning tasks. The chip is equipped with 32 GB HBM2e and its performance is up to 210 Tops. Trn1 instances combine up to 16 of these accelerators, linked by the NeuronLink interconnect (768GB/sec), have EFA connections at speeds up to 800Gb/sec (twice the speed of current GPU instances) and offer up to 8TB NVMe SSDs.

AWS allows you to cluster tens of thousands of Trainiums, connected by a petabyte-class network and connected via Amazon FSx to petabyte Luster storage. All of this allows you to quickly train giant models. For the job, a proprietary set of Neuron SDK tools is offered (same as for AWS Inferentia), but popular frameworks like Tensorflow or PyTorch are also supported. Trn1 is currently in preliminary access upon request.

AVnews