Aligned, AMD, and USC ISI Collaborate on MEGALODON Language Model

Aligned has announced a strategic collaboration with AMD and the University of Southern California's Information Sciences Institute (USC ISI) to accelerate the development of the MEGALODON language model, announced in a press release. This partnership focuses on utilizing AMD's Instinct MI300 GPUs and Aligned's AI expertise to train the model efficiently on non-NVIDIA architectures.

MEGALODON is USC ISI's flagship large language model, featuring a novel Moving Average Equipped Gated Attention (MEGA) architecture designed to improve long-context retention while reducing computational complexity. The collaboration aims to push performance boundaries beyond traditional CUDA-based environments by leveraging AMD's ROCm platform as a CUDA alternative.

Aligned will optimize MEGALODON's training on AMD's Instinct MI325X GPUs, providing the necessary tools and engineering support to USC ISI's researchers. This effort is part of a broader initiative to enable powerful AI on diverse hardware platforms, showcasing the potential of AMD's GPU infrastructure in large-scale AI training.