Constellation Research held a call with its Business Transformation 150 executives to talk shop, 2024 goals, generative AI and pressing issues such as Broadcom's purchase of VMware.
These gatherings, held under Chatham House rules, are a venue to share information and emerging trends. Here's a look at the topics for our January meetup.
Generative AI and challenges moving to production
Generative AI projects are top of mind for CXOs in the Business Transformation 150, but there are multiple concerns. Here's a look at some of the challenges:
- AI projects now have budgets and have gone from proof of concept and enterprises must prove returns. Cost takeouts, safety, regulatory compliance and efficiency are driving generative use cases.
- Concerns about the quality of data sets going into large language models (LLMs) abound. We are about to find out that data on the Web is far from open and that'll have an effect on model training. Enterprise insights and information will be hidden in the background.
- Transparency into model training data and the ability to select sources are lacking. Models are so complex that it is nearly impossible to follow lineage.
- Using an LLM for a proof of concept is one thing, but scaling it has legal concerns that will have to be baked into vendor contracts.
- Enterprises are likely to have more models than they assume today given moves toward smaller, more focused models designed for enterprise use cases.
- CEOs are wary of taking a recommendation from a decision engine or model without knowing what goes into the LLM and accuracy rates. Bias is another big concern for enterprises using LLMs and generative AI. It doesn't matter how much a company invests in AI if you can't trust the systems.
Related:
- Here's why generative AI disillusionment is brewing
- Privacy, data concerns abound in enterprise, says Cisco study
- Why vendors are talking RPOs, pipelines, pilots instead of generative AI revenue
- Generative AI features starting to launch, next comes potential sticker shock
Generative AI use cases
Our BT 150 CXOs were upbeat about healthcare generative AI use cases and see specialized models being a boon to diagnosis and augmenting human clinicians. There's an opportunity for AI to expand access to care. Google Research's AMIE, an AI system based on an LLM and optimized for diagnostic reasoning, was seen as a promising development.
Current use cases from our CXO panel include:
- Generative AI to augment radiologists and oncologists around the world. These models won't replace humans but will eliminate grunt work. Humans won't be able to compete with the speed of generative AI as models improve.
- Developer productivity was a big use case to automate code, improve processes and drive efficiency. Product quality was another developer use case.
- Marketing content, processes and efficiency.
- Combining generative AI with robotics process automation (RPA) so data from workflows can enable models to learn on the fly.
- Surfacing knowledge from data stores on product, technical specs, catalogs and unstructured data. One enterprise is trying to pull together all tech support emails and chats to build an internal knowledge base that can be mined. Generative AI can also summarize the unstructured data.
- Document classification for unstructured data with little metadata.
- Generative experiences leveraging enterprise data from core systems and content. Generative AI would generate widgets and materials that could be pulled into a sales or support conversation.
- Using AI to organize employee skills and knowledge based on natural language processing. Generative AI is also being used as a time saver to write performance reviews.
- Sales talent assessments using generative AI to pull together learnings and benchmarks for sales training.
- Automated testing processes for all that generative AI code. It's unclear what it means for testing and transparency of code.
- Order processing and using AI to put them into a structured format.
Constellation ShortList™ Artificial Intelligence and Machine Learning Cloud Platforms | Get ready for a parade of domain specific LLMs | Trust In The Age of AI | How much generative AI model choice is too much?
Enterprise buyers pan Broadcom's purchase of VMware
Broadcom's purchase of VMware is complete and there's a shift to subscription-based pricing. CIOs were actively looking at alternatives with many looking to move off VMware. Nutanix appears to be the biggest beneficiary. Among the VMware concerns:
- CIOs weren't surprised by Broadcom's move and one exec noted that peers are seeing price hikes of 100%. Broadcom made similar moves when it acquired CA and Symantec, CXOs said.
- Without a major change in strategy, it's hard to rip and replace VMware, but many companies are looking at jumping. One catch is that VMware still owns a lot of EMC's old big data portfolio and it's difficult to migrate.
- Enterprises took the time to plan a move off VMWare while the Broadcom purchase was delayed by regulatory issues. It will take a few years to move off VMware completely.
- Nutanix was gaining share and some CXOs were looking at open-source options for their Linux stacks.
- CXOs are expecting VMware's innovation, support and service to all decline--especially for enterprises that fall out of the top 100 or 200 clients.
