Maybe check out a film, or a book in French?
For a while there, we actually started watching "Peppa Pig" translated into French on Youtube. Not the most challenging material, but it was satisfying to be able to understand some of what was being said!
I was working at the bookstore at the time, and my co-workers thought I'd lost it when we got a shipment of Peppa toys and I started speaking to them in French.
Ha! That's great. I think "not the most challenging material" is actually a lot better than really hard stuff. Because that usually just brow-beats you into thinking you don't understand anything.
Would they have been less bothered if you spoke to them in English, I wonder? Also, this is assuming they believed you to be sane to begin with. Bold statement :P