Moreh Shows Off Fast LLM Inference on AMD GPUs at AI Event

The tech company Moreh recently showed off its distributed inference solution, the MoAI Inference Framework, running on AMD graphics processing units (GPUs) at a conference in San Francisco. This demonstration highlighted how well the framework works with large language models on systems equipped with 32 AMD Instinct MI300X GPUs across four nodes.

At this AI event, Moreh presented a live test of its GLM-5.1 LLM, which is powered by the MoAI Inference Framework. Visitors got to see and use the chatbot themselves, and they gave feedback on how fast it responded and how well it performed in real-world situations. This hands-on experience let attendees verify that the framework works as promised.

The demonstration showed key performance metrics in real time, like GPU usage, tokens per second, time to first token, and time per output token. This level of transparency helped visitors understand what the system can do and how well it performs. People at the event praised the live deployment of a big model on AMD GPUs with production-grade service levels.

Global leaders in AI and companies that buy these services were very interested in the system’s fast response times and stable performance. Moreh’s MoAI Inference Framework is seen as one of the first distributed inference solutions built specifically for the AMD ecosystem, which helps solve a major problem: the cost of running big AI models.

The CEO of Moreh, Gangwon Jo, said that this event gave customers around the world a chance to see firsthand how well large language model inference can be done on AMD GPU environments. He added that Moreh will keep working on its AI infrastructure software so enterprises can run their services as efficiently as possible, no matter what kind of GPUs they’re using.

Moreh’s distributed inference and computing technologies are designed to make running big AI models much cheaper, which should help more people use these systems around the world. This technology addresses a major challenge in the industry by making it easier to set up an efficient system for inferring things from data. Moreh also develops its own engine for AI infrastructure.

Moreh is strengthening its presence in the global market through partnerships with big tech companies, including AMD and Tenstorrent. The company’s innovative approach to running distributed inference on AMD GPUs marks a significant step forward for the industry.