Generative AI is Still Too Expensive for Many Use Cases

Lost in all the enthusiasm about generative AI are the costs to create and use models. Basically, the cost buckets include the initial cost of training a model; then the cost of supporting queries (prompts); and also the cost of the server infrastructure to support the operations. 

So far, concrete answers about how to monetize generative AI, beyond the generic “advertising, subscriptions, pay-per-use or licensing models have been proposed. 

To be sure, focused smaller models of the sort a single firm, in a single industry, might consider, to support customer service, marketing, sales or product development, for example, are relatively low, compared to the cost of training and using huge models, as in a search application, for example. 

Parameter Size	Training Cost	Prompt Cost	Infrastructure Cost
100 million parameters	$10,000-$100,000	$10-$100	$1,000-$10,000
1 billion parameters	$100,000-$1 million	$100-$1,000	$10,000-$100,000
10 billion parameters	$1 million-$10 million	$1,000-$10,000	$100,000-$1 million