Generative AI is Still Too Expensive for Many Use Cases

Lost in all the enthusiasm about generative AI are the costs to create and use models. Basically, the cost buckets include the initial cost of training a model; then the cost of supporting queries (prompts); and also the cost of the server infrastructure to support the operations. 

So far, concrete answers about how to monetize generative AI, beyond the generic “advertising, subscriptions, pay-per-use or licensing models have been proposed. 

To be sure, focused smaller models of the sort a single firm, in a single industry, might consider, to support customer service, marketing, sales or product development, for example, are relatively low, compared to the cost of training and using huge models, as in a search application, for example. 

Parameter Size	Training Cost	Prompt Cost	Infrastructure Cost
100 million parameters	$10,000-$100,000	$10-$100	$1,000-$10,000
1 billion parameters	$100,000-$1 million	$100-$1,000	$10,000-$100,000
10 billion parameters	$1 million-$10 million	$1,000-$10,000	$100,000-$1 million

Generative AI Training Costs

It is fairly obvious why generative AI is going to increase the need for computing resources: training an algorithm requires identifying patterns and structures within any data set. And that can be quite complex, depending on the number of parameters to be sampled, and parameters can run into the billions. 

In addition to the number of parameters, the sheer volume of data makes a difference, observers note. Specialized chatbots are less compute intensive while natural language translation can be quite compute intensive. 

Use Case	Data Set Size	Estimated Training Cost
Chatbot (small)	100 GB	$10,000
Chatbot (large)	1 TB	$100,000
Image generator (small)	100 GB	$10,000
Image generator (large)	10 TB	$1,000,000
Text summarization (small)	100 GB	$10,000
Text summarization (large)	100 TB	$10,000,000
Natural language translation (small)	100 GB	$10,000
Natural language translation (large)	1 PB	$100,000,000