The problem is that the diversity of a company is measured based on the employees of the company, not by the pool of candidates that were considered. Diversity goals are measured based on company make-up. Unfortunately, considering a diverse applicant pool does not always lead to a diverse result. This is what I was getting at by talking about that built-in assumption. I agree that a company that does not discriminate widens its pool of applicants and will result in better results overall. Given that, this seems like a problem that would take care of itself as companies that consider a diverse work force will do better than ones that don't (or don't do a good job of it).
I guess in summary I would say that considering a diverse applicant pool is absolutely the right thing to do and will result in better results. However, laws, regulations, and policies that attempt to impose diversity goals result in "special" rights, not "equal" because of how diversity is measured if for no other reason. Diversity goals that are measured based on the make-up of a company are just quotas by another name.