People may choose to trade with someone in a country that has low wages instead of trading with you because their choices are limited by the amount of money that they have.

Taxation is one of the causes of unemployment. If the government takes money out of the economy and doesn't put enough back in through spending (such as when they have a budget surplus), then there isn't enough money circulating around for people to buy all of your goods/services/labour.

In light of this, I think it's fair to say that these people are victims.