I'm already trying small LLMs on my computer. They're doing well!
My Computer is not strong, and while it takes a bit long to process, and the quality isn't always good. I'm sure with some tweaking, we'll see a model that gives better results than GPT4 and is able to run on average mobile phones.
It's not unrealistic!
Life might get easier as Hugging Face and others focus upon that sector.
I have a feeling it is going to be huge as the goal is to get this stuff working on mobile phones.