"Sons of God" also refers to holders of the priesthood, they did not need to be fallen angles, just men who forsook their covenants to go after worldly women.... but that a whole new topic.