It's not so difficult. It's just work, only work.
Grammar is not very hard. Vocabulary was the most difficult for me. And trying to think like a Japanese too. The way the Japanese make the sentences was sometimes weird for me.