From Smart to Genius: How GenAI is Elevating Your Smartphone Experience

In the future, smartphones will only become smarter with generative AI, which can create new context using natural languages like English or Hindi. Canalys expects Samsung Galaxy S24 to launch on 17 January with AI features like Live-Transcribe, but says that most new AI solutions and features will be rolled out as software updates in the future.

According to research firm Counterpoint, the year 2024 will be pivotal for shipments of Generative AI (GenAI) smartphones, with preliminary data predicting over 100 million units. Counterpoint predicts GenAI smartphone shipments will reach 522 million units by 2027, growing at an 83% compound annual growth rate. Consequently, between CY2024 and CY2027, GenAI smartphones are expected to be shipped over 1 billion times cumulatively.

Recently, chipset makers such as Qualcomm, Mediatek, Samsung and Google have focused on improving neural processing units and tensor processing units (NPU/TPU) performance. As AI accelerators, both NPUs and TPUs fall into this category. Machine learning (ML) workloads are accelerated by Google’s custom-developed application-specific integrated circuits (ASICs).

Similarly, NPUs are dedicated to handling AI/ML workloads. AI/ML algorithms are also being implemented locally by smartphone manufacturers such as Apple, Huawei, vivo, and Xiaomi in order to improve imaging quality, battery life, and typing experience.

However, integrating large language models (LLMs) and other GenAI models on-device requires a different computing platform. Moreover, Canalys recommends a dedicated processing unit for AI-related tasks as part of a smartphone’s system-on-chip (SoC). LLMs and other generative AI models should also be able to run on smartphones.

Further, the inference performance of an LLM on-device (allowing the model to generate responses that are relevant and appropriate based on the context) should be faster than the average adult’s reading speed, while the image generation time should be less than two seconds when using on-device AI. Generally, GenAI-enabled smartphones run natively-optimized AI models and have certain hardware specifications, according to Counterpoint.

Neural Engines or NPUs are present in most new iPhones and iPads. The researchers at Apple claim to have overcome the challenge of running large language models on devices with limited memory capacity with ‘LLM in a flash: Efficient Large Language Model Inference with Limited Memory’. Two techniques are described in the manuscript: ‘windowing’ and ‘row-column bundling’, which “together contribute to a significant reduction in data load and an increase in memory efficiency”. It is also reported that Apple has been developing a proprietary LLM named Ajax since early 2023 to integrate AI into its core apps and services, such as Siri, Apple Music, and Apple Watch.

On Qualcomm’s part, the company claims that with the Snapdragon 8 Gen 3 processor (likely to be used by the Samsung Galaxy S24 series), LLM models can run up to 20 tokens per second — one of the fastest in the smartphone industry — and generate images in a fraction of a second with Stable Diffusion (a popular text-to-image generative AI model that can create photorealistic images within tens of seconds given any text input). A number of brands, including ASUS, Honor, Nubia, OnePlus, Oppo, Realme, Sony, Xiaomi, and ZTE, have committed to integrating this chip into their forthcoming premium phones. Qualcomm is required to integrate AI into the devices, however.

In spite of this, some GenAI features are already being included in smartphones by big tech companies. Its advances in generative AI technology were showcased at Google’s I/O 2023 conference in May, allowing Android phones to be customized in a new way. As an example, Magic Compose – a new Messages by Google feature powered by generative AI – can suggest responses based on the context of the message, and rewrite it in a unique way. Google Photos (Pixel 8 and Pixel 8 Pro) now includes Magic Editor, a new experimental editing feature that uses generative AI to reposition, resize, and add fun backgrounds. Howling winds and loud crowds can be reduced in your video using Audio Magic Erase.

A further feature of Call Screen is that it uses artificial intelligence to reduce spam calls. A more natural-sounding voice will be used to answer calls from unknown numbers. In addition, it is smart enough to separate unwanted calls from those you want. Soon, Call Screen (similar to Samsung’s Smart Call function, which lets you know who’s calling even when the number isn’t on your contact list) will suggest contextual responses without you having to answer.

Using Stable Diffusion on smartphones is just the beginning, according to Qualcomm, which aims to create an AI stack that works on multiple devices. Thus, Qualcomm Technologies’ optimizations for Stable Diffusion can also be used for laptops, extended reality (XR) headsets, and any other device powered by Qualcomm Technologies. 

Due to the high cost of running AI processing in the cloud, Qualcomm stresses the importance of efficient edge AI processing. MediaTek’s Dimensity 9300-based devices will likely catch up with Qualcomm’s smartphones over the next two years, according to Counterpoint.