Saturday, March 2, 2024

MWC and AI Smartphones

Mobile World Congress was largely about artificial intelligence, hence largely about “AI” smartphones.


Such devices are likely to pose issues--or create opportunities--related to processing tasks and memory on devices; use of edge computing or cloud computing resources. That, in turn, might create an opening for different ways of envisioning device and service business models.


Regarding on-board resources, on-board machine learning models might require more on-device memory.  to load up even before we get to running them, although the availability of compressed models surely is coming. 


Processing also will be an issue. Running an ML model arguably requires more unique arithmetic logic blocks than your typical CPU, so specialized processors are likely necessary.


Smartphone processing likely will continue to be constrained by power consumption and heat generation limits, as well, so there will be some limits on on-board processing power. 


Leveraging cloud or edge computing obviously is a potential solution. Processing of some tasks--such as real-time language translation; camera features and voice-to-text will continue to make sense as an “on-board processing” capability. 


Other features might continue to make sense as an “edge- or cloud-supported” capability. 


The issues are that this could reshape needs for end-user data plan features and higher-speed, latency-bounded networks. Roaming costs also are an issue. 


So even if on-board processing is, in principle, ideal, it might not be practical for all devices (mid-range and low-end devices, for example). And heat and processor cost issues must be considered as well. 


As a marketing issue, “subscription phones and service” might need a rethink. To some extent, consumers often take advantage of subsidized phone offers from their service providers. So a service plan with a two-year contract that includes the cost of the device in the recurring cost already is a form of “phone as a service.”


Subscription plans for advanced AI service (Google’s Gemini or Microsoft’s Copilot) already exist. So we might see a rethink of possible product bundles that include, on a subscription basis, the device, the AI capabilities (on-board plus cloud or edge) and the recurring service plan costs. 


Creating such bundles should be easier for consumers to understand once we develop more valuable AI-enhanced apps and features usable on smartphones. People might expect AI features such as camera performance or image editing and translation services to be “bundled” with the device. 


But, eventually, some compelling additional use cases could--and should--develop that require an AI service plan that relies on cloud and edge computing, faster connections and more data allowances. So think of a 5G service plan using mid-band spectrum (for speed); unlimited data usage (so the external cloud and edge processing can be used) plus “AI device” supplied on a subscription basis, with a “new” device supplied every two or three years. 


Aside from all the practical details of figuring out the service provider’s cost to do so, we still need some new “killer apps” that make the purchase of an AI service plan such as Gemini or Copilot a reasonable and necessary investment by the consumer. 


As a business problem, this is a “logical bundle” issue. What features (device, AI, recurring service cost, features and apps) will make sense for many customers when all of those features are a subscription, not just the mobile service plan and the AI? 


Right now, it is not so clear what the new killer value requiring AI--and therefore a more-powerful device plus remote processing--is the trigger. Still, once one or a few such use cases do develop, with high customer interest, it will be easier to conceive of, and sell, new bundles of device, AI and service, for one recurring monthly price. 


Tuesday, June 6, 2023

Generative AI is Still Too Expensive for Many Use Cases

Lost in all the enthusiasm about generative AI are the costs to create and use models. Basically, the cost buckets include the initial cost of training a model; then the cost of supporting queries (prompts); and also the cost of the server infrastructure to support the operations. 


So far, concrete answers about how to monetize generative AI, beyond the generic “advertising, subscriptions, pay-per-use or licensing models have been proposed. 


To be sure, focused smaller models of the sort a single firm, in a single industry, might consider, to support customer service, marketing, sales or product development, for example, are relatively low, compared to the cost of training and using huge models, as in a search application, for example. 


Parameter Size

Training Cost

Prompt Cost

Infrastructure Cost

100 million parameters

$10,000-$100,000

$10-$100

$1,000-$10,000

1 billion parameters

$100,000-$1 million

$100-$1,000

$10,000-$100,000

10 billion parameters

$1 million-$10 million

$1,000-$10,000

$100,000-$1 million

Monday, June 5, 2023

Generative AI Training Costs

It is fairly obvious why generative AI is going to increase the need for computing resources: training an algorithm requires identifying patterns and structures within any data set. And that can be quite complex, depending on the number of parameters to be sampled, and parameters can run into the billions. 


In addition to the number of parameters, the sheer volume of data makes a difference, observers note. Specialized chatbots are less compute intensive while natural language translation can be quite compute intensive. 


Use Case

Data Set Size

Estimated Training Cost

Chatbot (small)

100 GB

$10,000

Chatbot (large)

1 TB

$100,000

Image generator (small)

100 GB

$10,000

Image generator (large)

10 TB

$1,000,000

Text summarization (small)

100 GB

$10,000

Text summarization (large)

100 TB

$10,000,000

Natural language translation (small)

100 GB

$10,000

Natural language translation (large)

1 PB

$100,000,000

Saturday, June 3, 2023

Will AI Boost Productivity Rates, and How Long Will it Take?

The percentage of enterprise capital investment on technology, compared to other physical hardware, has climbed since 1980, with technology spend reaching half of total spending by about 2020, according to the Wells Fargo Institute. 


And, according to figures from the U.S. Bureau of Economic Analysis, non-farm productivity has reached an annual peak rate about seven years after a wave of innovation was launched, assuming one believes knowledge or service work productivity can be quantified. Still, the point is that enterprises invest based on the expectation of higher sales and profits, made possible by productivity improvements. 


source: Morgan Stanley Research, Bureau of Economic Analysis   


Many believe a new era of investment is coming, driven by artificial intelligence; extended, virtual or augmented reality and the internet of things, that could boost productivity rates again. 


source: Morgan Stanley Research, Bureau of Economic Analysis  


If that happens, it might not be driven by efficiency gains, but by creation of new products and services. 


Decade

Digital Investment Trends Begin

Stated Objectives

1980s

Personal computers, mainframes, and minicomputers

Improve efficiency and productivity

1990s

The internet, e-commerce, and web-based applications

Reach new markets, improve customer service, and reduce costs

2000s

Cloud computing, mobile devices, and social media

Increase agility and flexibility, improve collaboration, and reach new customers

2010s

Artificial intelligence, machine learning, and natural language processing

Automate tasks, improve decision-making, and personalize experiences

2020s

Quantum computing, augmented reality, and virtual reality

Create new products and services, solve complex problems, and transform industries


MWC and AI Smartphones

Mobile World Congress was largely about artificial intelligence, hence largely about “AI” smartphones. Such devices are likely to pose issue...