Niv-AI exits stealth to wring more power performance out of GPUs

Electricity is a key raw material for artificial intelligence, but new processing techniques outstrip the ability of data center operators to manage their relationship with the power grid, forcing them to throttle down by as much as 30%.

“There is so much power squandered in these AI factories,” Nvidia CEO Jensen Huang said during a keynote speech at the company’s annual GTC customer conference. “Every unused watt is revenue lost,” the company proclaimed during the annual presentation.

Today, startup Niv-AI has emerged from stealth with $12 million in seed funding to solve this problem by precisely measuring GPU power use with new sensors and developing tools to manage it more efficiently.

The Tel Aviv-based startup was founded last year by CEO Tomer Timor and CTO Edward Kizis, and is backed by Glilot Capital, Grove Ventures, Arc VC, Encoded VC, Leap Forward, and Aurora Capital Partners. The company declined to share its valuation.

As frontier labs operate thousands of GPUs in concert to train and run advanced models, there are frequent, millisecond-scale power demand surges as the processors switch between computation tasks and communicating with other GPUs.

These surges make it difficult for data centers to manage the power they draw from the grid. To avoid being left without sufficient electricity, data centers pay for temporary energy storage to cover surges, or throttle their GPU usage. Both cases reduce the return on investments in expensive chips.

“We just can’t continue building data centers the way we build them now,” said Lior Handelsman, a partner at Grove Ventures who sits on Niv’s board.

The first step in Niv’s roadmap is understanding what’s going on; the company is now deploying rack-level sensors that detect power usage at the millisecond level on GPUs that it owns and alongside design partners. The goal is to understand the specific power profiles of different deep learning tasks, and develop mitigation techniques that allow data centers to unlock more of their existing capacity.

Naturally, the engineers expect to build an AI model on the data they collect, with the goal of training it to predict and synchronize power loads across the data center — a “copilot” for data center engineers.

Niv-AI expects to have an operational system in a handful of U.S. data centers in the next six to eight months. It’s an attractive idea as hyperscalers trying to build new data centers face difficult land-use and supply chain hiccups. The founders see their ultimate product as a missing “intelligence layer” between data centers and the electrical grid.

“The grid is actually afraid of the data center consuming too much power at a specific time,” Timor told TechCrunch. “The problem we’re looking at is a problem with two sides of the rope. One is to try to help the data centers utilize more GPUs, and hopefully make more of the power that they’re already paying for. On the other hand, you can also create much more responsible power profiles in between the data centers and the grid.”