Yes, I meant the block producers.

If it is not a permissionless network you will always need to delegate some entities that at the very least process transactions (in EOS's case, these are the BPs). The problem with this is that it leads to a rule of the rich. Sure, they can be voted in and out of power in theory, but in effect the network is not decentralized enough for this to work. Average Joes buying 10/100/1k EOS each cannot compete with organizations that own a considerable percentage of the total supply. In such a system there always will be whales, so we need to build systems where the system itself prevents the centralization of power.
Currently, there is a huge trade-off because the more decentralized the network is, the harder it is to scale to millions and billions of users. We'll get there though.