Generative AI with Red Hat AI

Red Hat AI increases efficiency in AI deployment and operations by providing access to enterprise-grade, open source Granite models. These small language models can be tailored to fit your needs. They are less costly, more efficient, and optimized for inference. They’re also covered by our Open Source Assurance program. 

Apply LLM compressor algorithms to further reduce hardware costs, and run them on hardware accelerators of your choosing. 

Technologies like vLLM are available to optimize the power of GPUs. This lowers hardware costs and improves token generation speed, leading to quicker responses in real-time applications.

Designed for accessibility, Red Hat AI provides a consistent user experience for data scientists, data engineers, application developers, and DevOps teams. This means better collaboration, fewer errors, and faster time to market. 

Red Hat AI includes access to InstructLab alignment tools, making it accessible for all your employees to contribute to a large language model (LLM). 

You can also customize your LLMs with retrieval-augmented generation (RAG). Red Hat AI provides capabilities like data ingestion, embeddings, and model evaluations, so you can produce responses that align with your specific business needs.

Red Hat AI provides users with the flexibility to choose where to train, tune, deploy, and run models and gen AI applications–on premise, in the public cloud, or at the edge. By managing your gen AI models within your environment of choice, you can control access, automate compliance monitoring, and enhance data security. 

vLLM supports flexible deployment of your gen AI applications by breaking up the work of processing across multiple GPUs. This distributes services across nodes that receive, process, and transmit data and makes for more efficient use of computing resources. 

Red Hat AI also supports disconnected and air-gapped environments, so you can safeguard your most sensitive data.

Our engineering team is dedicated to helping you navigate our AI platform. From the operating system to the individual tools, we can provide the help you need to move your AI strategy forward.