In a significant stride towards optimizing AI performance, Penguin Solutions Inc. (Nasdaq: PENG) announced the expansion of its OriginAI® portfolio on March 16, 2026. The latest solutions are designed to address critical challenges in GPU memory, particularly in relation to context size, concurrency, and the low latency requirements of enterprise-scale AI inference. This enhancement aims to accelerate time to value for organizations while mitigating performance limitations often associated with AI inference processes.

2. Key Features of OriginAI Inference Solutions

OriginAI inference solutions integrate seamlessly with NVIDIA RTX PRO 6000 and NVIDIA B300 GPU designs, facilitating the addition of large memory appliances. This integration is crucial for improving operational metrics such as GPU utilization, deployment velocity, and infrastructure reliability, thereby empowering organizations to execute AI workloads with predictable performance at scale.

The Experience Behind the Innovation

Drawing on over 30 years of expertise in memory solutions and more than 3.3 billion hours of GPU runtime, Penguin Solutions leverages this vast experience to deliver production-level inference. The company emphasizes that memory capacity and availability are just as vital as GPU compute power when it comes to affecting latency, system throughput, and overall user experience.

3. Insights from Leadership

Phil Pokorny, Chief Technology Officer at Penguin Solutions, expressed the company's commitment to optimizing AI inferencing. He stated, “Organizations must understand the factors that impact inference performance—which differ significantly from training—to productize AI and deliver accurate and fast outcomes.” This highlights Penguin Solutions' focus on operationalizing AI in real-world applications, ensuring that organizations can convert AI innovations into tangible business results.

4. Enhanced Capabilities with MemoryAI KV Cache Server

One of the standout features of the new OriginAI solutions is the integration of Penguin’s MemoryAI™ KV Cache Server. This innovative server is designed to support key-value (KV) strategies by expanding cache capacity, which is essential for low-latency, high-concurrency inference. Compatible with the NVIDIA Dynamo framework, the MemoryAI KV Cache Server not only optimizes performance but also enhances cost-efficiency for the next wave of AI deployments.

Intelligent Management with ICE ClusterWare

Additionally, Penguin Solutions offers the ICE ClusterWare™ software, which serves as an intelligent management layer. This software turns validated hardware into a fully-tuned AI cluster, providing health monitoring and auto-remediation features. By ensuring sustained peak performance, ICE ClusterWare enhances data security in multi-tenant environments by isolating workloads and protecting sensitive information.

5. Tailored Solutions for Diverse Industries

The OriginAI portfolio is designed to meet a variety of customer needs, with specific configurations for different applications. The NVIDIA RTX PRO 6000-based architecture is particularly aimed at enterprise-class copilots, retrieval-augmented generation systems, and document summarization. Conversely, the NVIDIA B300-based architecture caters to large, shared services and frontier model hosting, ensuring massive memory bandwidth and scalability.

Industry Applications

The practical applications of OriginAI solutions span several industries:

Financial Services: The need for ultra-low latency in AI-driven applications for fraud detection and real-time trading is critical for maintaining security and optimizing trading opportunities.

Healthcare: AI applications in diagnostics and patient monitoring require minimal latency to provide timely and accurate insights, often in life-critical scenarios.

Retail: AI-driven systems for personalization and inventory management enhance customer engagement and operational efficiency.

6. Conclusion: Transforming AI with Penguin Solutions

As AI continues to reshape organizational efficiency, accuracy, and innovation, Penguin Solutions’ advancements in the OriginAI portfolio are aimed at meeting the evolving demands of various industries. With tailored solutions that address inference objectives and key-value strategies, the company is well-positioned to help clients achieve measurable results and capitalize on the transformative potential of AI.

To learn more about these innovative solutions, attendees can visit booth #1031 at the NVIDIA GTC AI Conference and Expo from March 16-19, 2026, in San Jose, California.

Original press release