If you want I can provide sources/pubmeds, but essentially those claims typically came from "health doctors" and not scientists, which usually did not have data to support it.
More recently there has been studies to support that I believe, however I've yet to come across one that didn't have the leading "scientist" or funder being a dairy association, so go figure I guess.

From what I've studied phyto-estrogens cannot link up completely and thus do not have the same effect as mammal estrogens, which can link up with our "parts."
I remember actually having soy as a boy really is irrelevant unless you have about 2gallons a day, then a small impact appears to begin, although they concluded it wasn't statistically significant.
However, little girls (and women too, but less-so) benefit from soy consumption as the estrogens while they don't latch-on they, uhm, do the shimmy in front of your "parts" thus blocking animal estrogens from entering the cells, thus protecting from puberty and more-so breast cancer, as well as overall balancing hormones better.

This is likely one reason Asian women start puberty later, as well as lower dairy consumption and weight, and do not grow as large of breasts till they become pregnant, when they do significantly, uhm, inflate.

Like I said I can dig up some studies for it if you want; that myth originally also because bodybuilders always hit on soy because...well that's a whole 'nother topic.
Essentially whey is a by-product of milk production and the whey industry shilled the fuck out of bodybuilders into believing soy gave them man-boobs even though nobody ever got man-boobs from soy protein on bodybuilding forums...and funnily enough people always got man-boobs from doing a program called "GOMAD" which stands for "Gallon of milk a day."
Heh, I forgot about that.