Welcome back to My Weird Prompts. I am Corn, and today we are tackling a topic that literally sits in everyone's pocket. It is the dream of the truly intelligent smartphone.
And I am Herman Poppleberry. It is great to be back in the studio, or well, back in our living room in Jerusalem where the magic happens. We have got a really meaty prompt today from our housemate Daniel. He was showing me his phone the other day, a really nice OnePlus thirteen, and he was talking about how the shift from the old school Google Assistant to these new Gemini two point zero models feels like a major turning point, but maybe not a finished one.
Right, Daniel was asking about the path to success for embedded AI at the edge. Specifically, how close are we here in early February twenty twenty-six to having agentic AI that can fully control a mobile device? He is looking at the hardware and software challenges of miniaturization. How do we make these models small enough to run locally without making them, well, less smart?
It is the ultimate engineering trade-off, Corn. For years, we have been used to AI being something that happens elsewhere. You send a request to a massive data center filled with liquid-cooled graphics processing units, and it sends back an answer. But as Daniel pointed out, if you want an agent to actually control your phone, to navigate your apps, or handle your private data without it leaving the device, that latency and privacy cost of the cloud becomes a deal-breaker.
Exactly. And Daniel brought up a great point about his own hardware. He has got sixteen gigabytes of random access memory on his device, which is the standard for a flagship this year, but he still feels that struggle when the AI tries to do something heavy. It makes you wonder, if we are shrinking these models so effectively for edge devices, why do we still need those massive, energy-sapping server farms?
That is such an insightful question. We should definitely dig into the why of that later. But let us start with the state of play. When we talk about agentic AI on a phone, we are moving past the chatbot phase. We are talking about Large Action Models, or LAMs. These are models that do not just predict the next word in a sentence, but predict the next click, the next swipe, or the next system call.
And that is where it gets tricky, right? Because an agent needs a high degree of reasoning. If I tell my phone to find a flight to London that fits my calendar and then message my brother the details, the AI has to understand my calendar app, the browser, and my messaging app. It has to maintain a long-term memory of what it found in step one to use it in step three. Usually, that kind of reasoning requires a massive parameter count.
It does, or at least it used to. What we are seeing in twenty twenty-six is the triumph of quantization and architectural efficiency. When Daniel asks how we overcome the hardware challenges, the answer is largely in the Neural Processing Units, or NPUs. If you look at the latest Snapdragon and Dimensity chips, they are pushing over one hundred and twenty tera operations per second. That is dedicated silicon just for these matrix multiplications.
But even with a fast NPU, you still have the memory bottleneck. If a model is too big to fit in that sixteen gigabytes of RAM Daniel mentioned, it has to swap to the slower flash storage, and suddenly your agent feels like it is thinking through molasses.
Right, which is why quantization is the hero of this story. For the listeners who might not be deep in the weeds, quantization is basically reducing the precision of the numbers the AI uses to think. Instead of using high-precision thirty-two-bit decimals, we are seeing models compressed down to four-bit or even one-point-five-eight-bit precision, what we call BitNet architectures. It is like taking a high-definition photograph and turning it into a very clever piece of pixel art. It takes up a fraction of the space, but if you do it right, the human eye, or in this case, the user experience, can hardly tell the difference.
But is that true for reasoning? I remember back in episode one hundred thirty-one, when we were looking at the roadmap for this year, we talked about how smaller models tend to hallucinate more when they are pushed. If I am trusting an agent to control my mobile device, a hallucination is not just a wrong fact, it is a wrong action. It might delete an email instead of archiving it.
That is a crucial distinction, Corn. The margin for error for a mobile agent is much thinner than for a creative writing assistant. To solve this, developers are using what is called speculative decoding. This is a fascinating technique where you have a tiny, super-fast model on the device that makes a guess at what the next action should be, and then a slightly larger, more capable model checks that guess. It is like a junior employee doing the work and a senior manager signing off on it instantly. It keeps the speed of a small model but the accuracy of a bigger one.
That makes sense. It is like a system of checks and balances right on the silicon. But Daniel also mentioned the struggle of getting these models to be conversational. He noticed that when you shrink a model, the first thing to go is often that fluid, natural language capability. It starts sounding more like a robotic command line and less like a helpful friend.
It is a balancing act. If you dedicate all your parameters to reasoning and app control, you lose the nuance of language. But here is where the software side comes in. We are seeing a move toward modular AI. Instead of one giant model trying to do everything, your phone might run a suite of specialized small language models. One is an expert at understanding your voice and tone, another is an expert at navigating the Android user interface, and they pass tokens back and forth. This is the Mixture of Experts approach, but miniaturized for the edge.
So it is like a tiny team of experts living in your pocket. I like that image. But let us talk about the control mechanism itself. Daniel mentioned two ways this is being explored in the developer world. One is vision-based, where the AI literally looks at your screen pixels just like you do, and the other is the system-level approach, using application programming interfaces or command line interfaces. Which one is going to win?
That is the million-dollar question. The vision-based approach is incredibly elegant because it does not require developers to change anything. If the AI can see the button, it can click the button. It treats every app as a universal interface. We talked about this in episode two hundred ten when we looked at how transformers were learning to walk and interact with the physical world. Applying that to a digital screen is the same logic.
But vision is computationally expensive, right? Processing a video feed of your screen in real-time at thirty frames per second just to find a back button seems like a massive waste of battery.
Oh, it is a huge battery drain. If you run a vision-based agent constantly, your phone is going to be a hand warmer within twenty minutes. That is why I think the system-level approach, while slower to roll out, is the long-term winner. This is where the operating system itself is rebuilt around the AI. Instead of the AI pretending to be a human finger on a screen, it talks directly to the app's core logic. This is what Apple is trying to do with their App Intents and what Google is doing with their Android Intent system.
But as Daniel noted, that requires integrations. And integrations are slow. We have thousands of apps. Are we really going to wait for every developer to add an AI hook to their niche weather app or their local grocery store tracker?
We might not have to. We are seeing the rise of what people are calling semantic kernels. The operating system can basically scan an app's code or its metadata and build its own map of what that app can do. It is like the AI is teaching itself how to use the app without the developer's help. It is not quite as clean as a direct API, but it is much more efficient than vision.
So, looking at where we are now in early twenty twenty-six, how close are we to that seamless experience Daniel is dreaming of? He wants to just speak to his phone and have it do anything, no menus, no fidgeting.
I think we are at the eighty percent mark. The hardware is finally here. If you have a flagship phone from this year, you have the compute power. The software is the final hurdle. We are seeing the transition from the AI being an app you open, like Gemini or ChatGPT, to the AI being the shell of the phone itself. By the end of this year, I expect the flagship experience will be voice-first by default.
It feels like a massive shift in how we relate to technology. It goes from being a tool we operate to a partner we direct. But I want to go back to Daniel's question about the big GPUs. If we can do all this on a phone, why is the world still building massive data centers? Why is there still a war for those top-tier server chips?
This is where we have to talk about the difference between inference and training. Your phone is great at inference, which is the act of using a pre-trained model to perform a task. But training those models, teaching them the vast complexities of human logic and language in the first place, still requires tens of thousands of GPUs working in parallel for months. Plus, we are seeing the rise of Deep Reasoning models like the successor to OpenAI's o-one. Those models use chain-of-thought processing that is still too heavy for a phone to do quickly.
Right, and there is also the matter of peak performance. A mobile device has a thermal envelope. It can only get so hot before it has to slow down to protect itself. A server in a climate-controlled data center does not have that problem. If you want the absolute cutting-edge reasoning, the kind that can solve a new physics problem or write a complex codebase from scratch, you still need the cloud.
Exactly. Think of it like this. Your phone is like a very smart personal assistant who knows your life perfectly. But the cloud is like a university research library. You want your assistant to be with you all the time, but sometimes they need to call the library to look something up. The future is hybrid. The agentic control stays on the device for speed and privacy, but the heavy lifting, the deep knowledge retrieval, that still happens in the cloud.
That makes sense. It also addresses the privacy concern. If my agent is navigating my bank app or my private messages, I want that logic to stay on my silicon. I do not want a video feed of my bank balance being streamed to a server just so an AI can find the transfer button.
Absolutely. That is the privacy-first path to success. And I think that is a huge part of why companies like Google and Apple are pushing so hard for on-device AI. It is not just about speed, it is about trust. If they can tell you that your agent is a black box that even they cannot look inside, that is a massive selling point.
So, for Daniel and his OnePlus, what is the immediate future? Should he expect an update this year that finally delivers on that full control?
I think he should look at the experimental projects he mentioned on GitHub, like the Open Interpreter mobile ports. We are seeing the community move faster than the big corporations in some ways. There are already frameworks that allow you to give an AI access to the Android accessibility layer. That is the secret sauce. The accessibility layer was designed for screen readers for the visually impaired, but it turns out to be the perfect map for an AI agent. It tells the AI exactly what is on the screen in a text format that is very easy for a model to understand.
That is brilliant. Using a tool meant for accessibility to empower an AI. It is a nice bit of technical irony. But Daniel also asked about the conversational capabilities. How do we keep the model from getting stupid as it gets smaller?
One word, distillation. This is a process where you take a massive, trillion-parameter model and you use it to train a smaller model. It is like a world-class professor distilling their lifetime of knowledge into a very concise textbook. The smaller model does not know everything the professor knows, but it learns the most important patterns and shortcuts. In twenty twenty-five, we got very good at this. The small models of today are actually more capable than the giant models of three years ago.
It is amazing how fast that curve is moving. I remember when we thought a seven-billion parameter model was the smallest you could go for anything useful. Now we have models with one or two billion parameters that are outperforming the old giants.
And they fit easily into the RAM of a modern phone. So, to Daniel's point, the hardware challenge is being met by more efficient software. We are not just making the engines bigger, we are making the cars lighter.
Let us pivot a bit to the practical takeaways for our listeners. If you are looking at the current landscape of mobile AI, what should you be looking for in your next device? Is it just about the TOPS count on the NPU?
That is part of it, but look at the memory bandwidth too. AI models need to move a lot of data very quickly between the memory and the processor. A phone with LPDDR5X or the newer LPDDR6 memory is going to run an agent much smoother than one that just has a high clock speed. And honestly, pay attention to the operating system's approach to permissions. An agent is only as good as what it is allowed to see.
That brings up a good point about the user experience. If my phone is constantly asking me for permission to see this app or that app, the magic of the agent disappears. It becomes a chore.
Right. We need a new permissions model for the age of AI. Instead of allow this app to access your photos, it might be allow your personal agent to manage your digital life. It is a much broader level of trust. We are going to see some growing pains there, for sure. There will be security researchers who show how an agent can be tricked into jailbreaking its own phone through a malicious website or a clever prompt.
Oh, I can see the headlines now. My AI Agent Emptied My Bank Account Because I Read a Weird Tweet. That is a scary prospect.
It is. This is why the path to success is not just technical, it is also about safety. We need guardrails that are as robust as the models themselves. But I am optimistic. The benefits of having a truly agentic assistant, something that can handle the digital drudgery of our lives, is just too big to ignore.
It really is. I mean, think about the time we spend just moving data between apps. Copying a tracking number from an email to a delivery app. Taking a date from a text and putting it in the calendar. If an agent can do that with a single voice command, we are reclaiming hours of our lives every week.
And that is the real goal. It is not about having a cool tech demo on your phone. It is about the technology finally getting out of the way. We have spent the last twenty years learning how to speak computer. We learned how to use mice, then touchscreens, then specific app layouts. Now, the computer is finally learning how to speak human.
That is a great way to put it. We are moving from the era of the user interface to the era of the user intent.
Exactly. And to Daniel's question about how close we are, I would say that by the end of twenty twenty-six, the idea of navigating an app will start to feel a bit old-fashioned. You will still do it for fun, like scrolling through social media, but for tasks? You will just state your intent and let the agent handle the navigation.
I am looking forward to that. Although, I might miss my brotherly debates with you if an AI can just settle all our factual disputes instantly.
Oh, I think we will always find something to argue about, Corn. The AI can give us the facts, but it cannot give us the perspective. That is still on us.
Fair point. Speaking of perspective, if you are listening to this and you have your own thoughts on where mobile AI is headed, or if you have tried some of those GitHub projects Daniel mentioned, we would love to hear from you. You can get in touch through the contact form on our website at myweirdprompts.com.
And while you are there, you can check out our full archive. If this conversation about hardware and NPUs piqued your interest, definitely go back and listen to episode one hundred forty-five where we talked about the war on the screen and how voice control is finally winning. It provides some great context for how we got here.
And hey, if you have been enjoying the show and finding these deep dives helpful, we would really appreciate it if you could leave us a review on your podcast app or on Spotify. It genuinely helps other people find us and keeps the show growing.
It really does make a difference. We see every review and we appreciate the support from our community.
Well, I think we have covered a lot of ground today. From quantization and NPUs to the philosophical shift from tools to agents. Daniel, thanks for the prompt. It gave us a lot to chew on.
Yeah, thanks Daniel. I am going to go back and look at those GitHub projects you mentioned. I might try to get a vision-based agent running on my old test phone tonight just to see how hot it actually gets.
Just keep a fire extinguisher handy, Herman.
Will do.
This has been My Weird Prompts. You can find us on Spotify and at myweirdprompts.com. Thanks for listening, and we will talk to you in the next one.
Stay curious, everyone.
So, Herman, before we totally wrap up, I had one more thought. We talked about the hardware and the software, but what about the data? These models at the edge, if they are going to be truly personal, they need to learn from us. Is that happening locally too?
That is the next frontier, Corn. It is called on-device training or federated learning. The idea is that your model gets smarter by observing your habits, but that learning stays on your phone. It might send small, encrypted updates back to a central server to help the overall model get better, but your specific data never leaves.
That feels like the final piece of the puzzle. A model that is not just smart, but smart about me.
Precisely. And that is why Daniel's sixteen gigabytes of RAM is so important. It is not just for running the model, it is for giving it room to grow and adapt. We are moving away from static software to living, breathing digital extensions of ourselves.
It is a wild time to be a script writer, I will tell you that.
It is a wild time to be alive, Corn.
True that. Alright, let us get out of here. I think I hear Daniel in the kitchen.
Probably asking why his phone is so hot.
Exactly. Catch you later, Herman.
Bye, Corn.
Thanks again for listening to My Weird Prompts. We will be back soon with more deep dives into the prompts that make us think. Check out myweirdprompts.com for the RSS feed and more. See you next time.
