Yes, you can. To do so you would have to physically alter the electronic circuit that enables the microphone to work. Almost no one has the capability to do this, because electrical engineering is difficult to understand and the tools necessary to alter phone circuitry is prohibitively expensive. It is entirely possible to make a circuit that turns the microphone on and off. That does not mean you can do it without undertaking the education and expense to do so.

I just don't carry a phone outside the house, which means I can only be tracked when I am detected by someone elses phone. I am aware that all my speech and text are captured when I use phones - even someone elses phone - so try to arrange for conversations I do not want surveilled to only happen in person, when no one has a cell phone in their possession. However, even then people I speak to can later recount those conversations using their phones, so my ability to limit the surveillance of my communications is extremely limited, mostly by the security operations of those I communicate with.