Highlights:

  • Startup d-Matrix, which raised USD 44 million in a round of funding, is one of the vendors developing technology to aid in accelerating AI inference for transformer models.
  • Project Bonsai’s first area of focus is industrial controls, which include chip manufacture and design.

The introduction of popular generative AI models has been one of the most popular trends in Artificial Intelligence (AI) this year. The number of startups and application cases have increased because of emerging technologies like DALL-E and Stable Diffusion.

One of the core technologies that generative AI relies on is transformer models. On the inference side, where systems forecast and develop the outcomes of a model, the use of transformers for generative AI and other use cases can be resource intensive.

Startup d-Matrix, which raised USD 44 million in a series A round of funding, is one of the vendors developing technology to aid in accelerating AI inference for transformer models. A round of funding in April to support the development of its AI accelerator hardware technology.

Microsoft has expressed interest in the company’s Digital In-Memory Computing (DIMC) technology, despite it being made available to the public.

Microsoft and d-Matrix revealed that the d-Matrix DIMC technology will be supported by Microsoft Project Bonsai reinforcement learning. The two companies anticipate that this will significantly speed up AI inference.

Kingsuk Maitra, the principal applied AI engineer at Microsoft, said, “Project Bonsai is a platform which enables our version of deep reinforcement learning, and we call it machine teaching. We have trained a compiler for d-Matrix’s one-of-a-kind digital in-memory compute technology, and the early results are very encouraging.”

What Project Bonsai by Microsoft is all about

Microsoft has been working on Project Bonsai for several years, and a preview version is now publicly accessible.

According to Maitra, the effort aims to remove the difficulties related to deep reinforcement learning networks. Project Bonsai’s first area of focus is industrial controls, which include chip manufacturing and design.

The capability of deep reinforcement agents getting trained to perform control tasks using a high-level language, created at Microsoft Project Bonsai called as Inkling, is part of the technology.

According to Maitra, deep reinforcement learning doesn’t require labeled data. Instead, it picks up new information from the environment’s feedback, which may be emulated with a simulator.

A trained Reinforcement Learning (RL) agent—referred as “brains” by Microsoft, is the outcome of a training loop. When used to accomplish the task, brains can do significant acts.

Maitra said, “We are running active real-life workloads and training the compiler, relative to those real-life workloads, most of them with well-known large language models with different Bonsai brains.”

In 2023, the d-Matrix Corsair will launch

There are no publicly accessible d-Matrix chips, but the first one known as, Corsair, is scheduled to arrive in 2023.

In an interview, Sudeep Bhoja, cofounder and CTO at d-Matrix, said, “We’re building an accelerated computing platform for transformers and specifically focused around generative AI.”

According to Bhoja, the chips that d-Matrix is developing can be built with a modular approach, bundled with a CPU, or integrated on a PCI card that connects to a cloud server.

With its DIMC technology, which offers high performance and low latency, the d-Matrix technology is intended to aid in speeding up AI inference.

Thanks to Microsoft’s Project Bonsai, d-Matrix now has a compiler that can create deep reinforcement learning tools for their silicon. Assisting to support continuous development and deployment of generative AI models is one of d-main Matrix’s objectives.

Bhoja said, “We want to enable [generative AI models] because it takes a lot of processing power, there are latency constraints, and it is user-facing. You have to be able to do it in a very energy-efficient way so that the data centers don’t have to bring in more power….”