How Users Can Prevent Data Breaches with Local LLMs on Smartphones
Flagship smartphones have long been known for their outstanding cameras, displays, and battery life. However, a new feature is setting the latest models apart from the rest: the ability to run large language models (LLMs) locally on the device. This capability could be a game-changer in how we interact with our smartphones and, importantly, how we secure our data.
What Are Local LLMs?
Large language models (LLMs) like GPT-3 and its successors have revolutionized how we use artificial intelligence, offering impressive capabilities in text generation, translation, summarization, and more. Traditionally, LLMs are hosted on powerful servers, requiring constant internet access to function. However, running LLMs locally on smartphones means users can leverage these powerful tools directly on their devices without having to send data to the cloud.
Key Announcements
A significant step was marked by Apple’s recent iOS 18 announcement, featuring Apple Intelligence for devices running the A17 Pro. Samsung has also been making strides, unveiling Galaxy AI for their flagship smartphones last year.
In the near future, your mobile phone will have AI LLM's built into the core operating system.
This means, you will search LESS online for information, it will be generated locally on your mobile phone.
This means traffic for search based content will shift DRAMATICALLY pic.twitter.com/T5ePlNMvrS
— Busy Works Beats (@BusyWorksBeats) July 10, 2024
Data from Canalys suggests that by the end of 2024, 16% of new smartphones will be generative AI-capable, a figure expected to rise to 54% by 2028. NVIDIA’s CEO, Jensen Huang, believes this technology will save energy and time, reducing our need to search for information online as it will be generated locally instead.
Why Run LLMs Locally?
One of the most significant benefits of running LLMs locally is enhanced data security. Cloud-based LLMs necessitate sending data to external servers, increasing the risk of data breaches and unauthorized access. Reports from HiddenLayer indicate that 77% of businesses experienced breaches in their AI systems over the past year, with third-party vendors contributing substantially to these risks.
When LLMs are run locally, data never leaves the device, reducing exposure to external vulnerabilities. This approach protects sensitive information such as personal identifiers, financial data, and business secrets.
Surprised by how many tech enthusiasts view the current AI revolution as merely another technological breakthrough.
We're actually experiencing an unprecedented historical shift that none of us have ever seen.
Here, a one-plus 24GB mobile running a Mixtral 8x7B at 11… pic.twitter.com/v6RlpkEfEn
— Rohan Paul (@rohanpaul_ai) July 3, 2024
Hardware Capabilities
Running LLMs locally indeed requires powerful hardware. Based on current advancements, only flagship hardware can efficiently run these models. For instance, the MLC Chat app, one of the easiest ways to run LLMs on smartphones, is currently available only for Samsung S23 with the Snapdragon 8 Gen 2 chip.
Even with smaller models like SLMs (small language models), you still need top-notch hardware. A user managed to run Mixtral 8x7B at 11 tokens per second on a OnePlus with 24GB of RAM, supported by PowerInfer-2, underlining the necessity for high-performance devices.
MediaTek Dimensity 9300: Real-time generative AI on your phone! #AI #generativeai #mediatekdimensity9300 pic.twitter.com/92Ya50ABCQ
— Android Authority (@AndroidAuth) July 1, 2024
Upcoming Hardware Options
To broaden accessibility, midrange System on Chips (SoCs) like the Qualcomm Snapdragon 7+ Gen 3 are in development. With a Qualcomm Hexagon NPU, these midrange SoCs aim to deliver improved AI performance and run generative AI capabilities locally, with less latency.
Leading SoC manufacturers continue to make strides. MediaTek’s Dimensity 9300 can execute text-to-video generation and real-time GenAI animation, featuring an APU 790 AI processor optimized for tasks like running Meta’s Llama 2. Arm’s recent unveiling of Cortex-X925 and Cortex-A725 CPUs supports a wide range of AI tasks efficiently, reducing the need for cloud-based computation.
This hardware evolution will incrementally allow more users to run LLMs locally, offering the benefits of AI without compromising data security.
Run locally.
Only way to keep your privacy.
Among other advantages. https://t.co/liK3HlsWmg
— Robert Scoble (@Scobleizer) January 24, 2024
Conclusion
Flagship smartphones are set to redefine user expectations by enabling local LLM capabilities. While we’re on the cusp of broader adoption, it’s clear that locally run LLMs will play a vital role in protecting personal and sensitive data. As hardware continues to advance, we may soon see these features trickling down to more accessible devices, making high-level AI accessible to all while ensuring user privacy.
For more detailed updates and the latest advancements, stay tuned, and follow the tech announcements closely.
also read:Introducing the Colorful and Modular Nothing CMF Phone 1: A Refreshing Take on Smartphones