oh, this topic is my pain.
Hard to believe but 10 years ago I was rather great in English and enough good in German.
English was my main foreign language in the university, and German was the second one (we studied it much less but still..). I could speak English fluently, but pronounciation was with heavy Russian accent because I had a lack of practice with native speakers. And I could speak German. Not confident and fast but I could say something, I passed my state exam in German laguage and studied my 25 exam tickeks by heart;)
Then I changed my activity in life and started to work only with my mother tongue Russian, and oops...very fast I lost ALL my German, and most of my English vocabulary. I still can read and understand English but it's much harder for me to say something:( Grammar is saved mosly, but vocabulary....:( now with the help of Steem and Hive I practiced it a little bit but still it's not that very level.. and actually I am sure it will never ge good without speaking with nativer speakers. I am at your stage until you didn't come to Japan. To be in any languahe you must speak it every day. With nativve speakers if it's possible otherwise....bye, skills!