peopleimages.com - stock.adobe.c
Inside Zoom’s generative AI strategy
Zoom’s federated approach towards generative AI allows it to use a combination of models that are most suitable for the task on hand while lowering costs for itself and customers
Zoom has been baking artificial intelligence (AI) into its products for some time now, enabling remote employees to see their co-workers up-close in hybrid meetings and delivering conversation insights to sales teams, among other capabilities.
In September 2023, the company took those capabilities to the next level with AI Companion, a generative AI (GenAI)-powered assistant that was designed to increase productivity, facilitate seamless collaboration and derive deeper insights for employees across the Zoom platform.
Besides providing workers with summaries of meetings they have missed and advising meeting participants on speaking skills, AI Companion can also help them gather materials from different sources ahead of a meeting. The tool is now being used by over 200,000 accounts, with some four million summaries created so far.
Underpinning AI Companion is Zoom’s federated approach towards GenAI, according to its chief product officer, Smita Hashim. “What that means is that we’re not just strapping one model – we work dynamically across multiple models to create results for the user,” she told Computer Weekly on a recent visit to Singapore.
“We have partnerships with OpenAI, Anthropic and Meta, but we also have our own model, and we will allow customers to bring their own models should they desire to do so.”
Hashim said this approach is more cost-effective as not every task requires the use of massive, large language models (LLMs). Huang Xuedong, Zoom’s chief technology officer, said in a recent blog post that Zoom will first employ a lower-cost LLM most suitable for the task on hand.
“Our Z-Scorer then evaluates the initial task completion quality, and if needed, we use a more advanced LLM to augment the task completion based on what was achieved from the initial LLM, similar to the way that a cohesive team working together is able to create a higher-quality product more efficiently than any one individual,” he said.
Read more about AI in APAC
- Organisations in APAC are deploying and experimenting with generative AI in healthcare, citizen services and other use cases amid cost-related concerns and other challenges.
- AI can be transformative in government, but its implementation has to be done in a way that drives positive outcomes while mitigating its downsides, according to the head of Australia’s Digital Transformation Agency.
- Culture Amp is building a generative AI capability that summarises employee survey responses, automating a process that typically takes HR admins up to hundreds of hours to complete.
- The multidisciplinary nature of AI offers career opportunities not only in builder roles like engineering and data science, but also in AI ethics and applied AI.
Hashim said a federated approach and the use of smaller models can deliver results with lower latency while enabling Zoom to provide AI Companion at no extra cost to customers with existing licences at a time when organisations are concerned about the cost of GenAI deployments.
On how Zoom is managing the costs of delivering GenAI capabilities to customers, she said the company’s technical innovation has given it a good margin for its products. “There is a cost to us, but we can do it in a pretty cost-effective way,” she said.
Addressing concerns about data privacy and security, Hashim assured users that Zoom does not use their data to train models and has published a whitepaper outlining the architecture, data storage details, retention periods and usage specifics. The company also provides a zero data retention option upon request.
Flexible architecture
Hashim said the flexibility of Zoom’s architecture could open opportunities for customers to run Zoom’s AI capabilities on-premise like they now can with some Zoom services through the Zoom Node platform that extends Zoom cloud functionality to customer datacentres.
“We are not running AI on [Zoom Node] right now, but we offer it for our Phone and Team Chat products,” she added. “The control plane is still running on Zoom, but these are possibilities as we move forward to address the needs of customers.”
While Zoom has a foundation in conversational intelligence, Hashim did not rule out the potential use of domain-specific models for industries such as healthcare, where customers may have connected the Zoom platform to other applications and data sources via application programming interfaces.
“In terms of domain-specific models, we’re having some of those discussions with multiple providers, and we are very willing for customers to bring their own models,” she said. “Some of that becomes easier because we have a federated approach which lets you plug in multiple models.”
Ricky Kapur, Zoom’s head of Asia-Pacific, said AI Companion has opened doors to customers in the region who are concerned about the cost of GenAI capabilities and want an alternative platform to ensure business continuity.
“I believe – and I’ve seen in a few cases – that it has accelerated the process of them bringing in Zoom as an alternative platform to the incumbent,” he said. “AI Companion provides additional value, and we see it bringing us additional revenue.”
