I'll answer this question using myself as an example and hope it fully conveys my views on the topic.

Up until my final year in high school, I hated mathematics. My very first maths teacher was unfriendly and taught the subject like he was being forced. Since he took me for three years, small wonder I grew to hate it.

Later on, I got another teacher who was, sadly to say, inept, and so my difficulty in the subject continued.

Luckily, I had a change of view when a different teacher took to teaching it in my final year. He was patient and experienced and this changed my views about the subject.

I believe this is the experience of most people, leading to them forming a wrong notion about the subject.