Zoho said it will build narrow use case focused language models for its platform on Nvidia after seeing a 60% increase in throughput and 35% reduction in latency compared to the open-source frameworks used previously.
The company, which offers a broad suite of business applications, has been building its AI stack and features in its portfolio.
Zoho said that it will use Nvidia's NeMo, a part of Nvidia AI Enterprise, and GPUs for its models. Zoho said it has spent more than $10 million on Nvidia technology and plans to invest another $10 million in the next year.
- Lessons learned from four Zoho customers
- Zoho Analytics revamp aims to connect, BI, data scientists, business users
- Zoho Advances Its Value-Leading Analytics and Business Intelligence Platform (research)
- Zoho CEO Sridhar Vembu: Long-term thought leadership on innovation, technology, people
- Zoholics 2024: Zoho aims to democratize CRM, infuse AI across platform, court developers, create cybersecurity stack
In a statement, Ramprakash Ramamoorthy, Director of AI at Zoho, said the company is focused on developing large language models (LLMs) designed for business use cases and integrated into its stack.
Zoho's focus has been on using smaller models that are more use case focused and cost efficient. Zoho, which uses multiple models, doesn't train its models on customer data.
According to Zoho, its LLM efforts will revolve around multimodal, vision and speech capabilities. The company said it is testing NVIDIA TensorRT-LLM.
Constellation Research analyst Holger Mueller said:
"This is a good move by Zoho and it's not surprising. There is no alternative to NVidia when it comes to on-premises AI. It’s a great validation of the Nvidia stack as Zoho tried alternate solutions, and does not shy away from stating that Nvidia is more efficient. The question is whether can a Nvidia stack deliver at an attractive SMB price point."
