Was the real school so small that it could only field six players for the team?

Not entirely. Similar to the fictional Hickory High in the film, it's true that there were only 161 students enrolled at the real school (Milan High). However, unlike in the movie, 58 of the 73 boys at the school tried out for the basketball team. There were 10 players on the Milan team in 1954, not six.