NTT R&D Forum 2024: New horizons for photonics-based chips, clouds & qubits

NTT’s Cognitive Foundation

“The goal here [with NTT’s Cognitive Foundation] is to optimally arrange and control the amount of computation according to the supply and demand of green energy. In this way, we aim to promote local production for local consumption of green energy and to improve the usage efficiency of renewable energy by dynamically arranging workloads based on supply-and-demand conditions of renewable energy,” said Kinoshita 

The company has noted that it is becoming increasingly difficult to extend GPU clusters at datacentres concentrated in urban centers due to a lack of rack space. Given that datacentres cannot be optimally arranged from a power requirement and data residency need, this problem is compounded. NTT says a large-capacity and low-latency APN service can optimally arrange computational resources and data and flexibly use GPU clouds.

“We have an exhibit that shows how a real application like LLM fine-tuning can operate with the same level of performance even at a remote base. This enables customers who use a GPU infrastructure to use multiple distributed datacentres and to deal flexibly and securely with changes in AI demand,” said Kinoshita.

In addition, Kinoshita says NTT aims to raise power efficiency even more by promoting an all-optical network even further so that it can economically expand APN areas. The company’s plan is to promote more enhancements in APN to further expand its use.

On-demand connections

One of these enhancements is “on-demand connections” – unlike leased lines that are essentially permanent, on-demand connections can be achieved in an end-to-end manner by automatically setting APN nodes from the controller at the time that the user needs those nodes.

“To use APN in an end-to-end manner, multiple wavelengths must coexist without colliding with each other,” said Kinoshita. “The technologies that will help meet this requirement are optical path design technology and wavelength conversion and wavelength-band conversion technology.”

In other areas of development, inter-chip-wiring power consumption is increasing.

At present, various types of cooling techniques are being used, but we can expect the cooling limit to eventually be exceeded. 

Compared to electrical wiring, a key advantage of using light for inter-chip-wiring is that power consumption does not increase even for long transmission distances on a circuit substrate. At NTT, Kinoshita says the company is promoting the evolution of silicon photonics along with the evolution of membrane photonics. Also here we see that downsizing optical devices and making them more efficient is essential to achieving small optical transceivers.

Generative AI/tsuzumi

“Since the announcement of the tsuzumi LLM in November of last year, we have provided many introductory consultations to more than 900 companies over a period of one year. Tsuzumi was the first LLM in Japan to be adopted in Microsoft’s Models-as-a-Service lineup,” said Kinoshita. “Training data size and context size are increasing, the German, Italian and Spanish languages are being supported in addition to Japanese and English… and multimodal support is also progressing.”

Response accuracy is increasing in this technology by enhancing retrieval augmented generation, or RAG.

“An AI agent operates a personal computer on behalf of the user. In daily work, it is rare for one task to be completed on one page. In the purchasing of goods, for example, the user would refer to general web pages when looking for a product, open up both a purchasing system and a web page when deciding on what product to buy and finally enter the information needed to make the purchase. With tsuzumi, however, only a chat session is needed to automatically open the required pages and enter the information needed,” he said.

Unlike past digital humans that make mechanical responses, NTT says it aims to develop a digital human that is capable of more human-like, gentle exchanges called “synlogue” based on the idea that the speaker and listener create utterances together instead of one utterance necessarily being completed by a single speaker.

To this end, it is researching and developing new dialogue architecture that creates a series of utterances through the cooperation of multiple LLMs having different processing speeds and specialties.

In this way, it hopes to build a digital human capable of more natural dialogue that can freely speak and easily be spoken to that will utter responses in agreement, create pauses in generating utterances by deliberately hesitating and let the conversation partner talk if interrupted while talking.

AI Constellation is NTT’s approach to deploying AI that offers diverse viewpoints instead of just a single solution thereby improving interpretability by a human. As a use case of AI Constellation, a workshop in which multiple AI agents discuss local social problems.

“We are developing technology for achieving “Human-AI cooperation” that merges the AI Constellation concept and the use of a person’s digital twin, gives AI agents individuality… and that enables AI to cooperate with humans to pursue creative and productive activities together as partners,” said Kinoshita, rounding out his presentation by also referring to why AI model development was originally the role of R&D, but NTT AI-CIX aims to provide a full range of services from consulting to AI model development and platform services by focusing on what kind of problems are present in the customer’s industry and how to go about solving them.