No, you don't lose anything, nor gain anything, by moving HBD in or out of your wallet. The correct interest accrues every block based on the balance that is there. It is only credited at most every 30 days though, but still tracked internally.