Sure, but in his notebook, he wrote: "it will be a girl".
So, if it is a boy, you'll remember that is what the doctor said.
But if it is a girl, you will tell the doctor that he was mistaken, and he will show you his notebook and tell you that you did not remember correctly his prediction.