Eric Newcomer

Databricks' AI Training Cost Doubled, Most Wasted

Back in 2024, when Databricks released its open-source DBRX model, the data warehousing company bragged that it had spent $10 million on the training run that made that model possible. Last week, in an expansive interview ahead of the release of a suite of new Databricks AI products, the company’s boisterous CEO, Ali Ghodsi, came clean: Databricks had actually spent about $20 million on the cluster to train the model. Only $4 million went to the training run that actually produced DBRX. The remaining roughly $16 million went to stumbling through the wilderness, downtime, and other mistakes of inexperience. They’d either underestimated or overestimated the cost depending on how you slice it. The waste felt unavoidable. “First of all the cluster is not running all the time — it’s idle,” Ghodsi told me. “You don’t just yolo and say hey let’s run the thing and then it comes out and it’s like after a month of training it’s like oh it was shit. So you do smaller training runs — so you work your way up to it.” Ghodsi estimated that even today many model providers are spending multiples of their reported training run figures just to get to the point where they can perform a high-quality training run. After the DBRX model was released, Ghodsi said, “That model is worth shit in three months.” Even old models from OpenAI have “zero value today,” he said. “It’s very rare in life that that is true.” He contrasted the depreciation of foundation models with software-as-a-service products. Outdated software typically slowly churn customers, delivering profits for years after its prime. In software, he said, “You have that asset that you’ve milked for four or five years.” However, he said, “in the LLM scenario, it just evaporates. So I think it’s not a good business for us to be in.” By the time Databricks released DBRX, Ghodsi says he knew he wanted out of the foundation model game. https://lnkd.in/gw-Xs2ua