Our God in the Bible is a jealous God. He doesn't tolerate we His children involving ourselves with other gods. We are either for God or not for him. God listens to the people who call on his name. People who go to idols are not going to God and calling on His name, so they shouldn't expect that God will listen to them.
Those who practice idol worshiping are turning their backs against God. We are meant to serve the one and only living God which is the God of the Bible. Idols are powerless, they don't have any power to carry out anything or the ability to answer prayers. Idols are nothing but the handworks of men and the creations cannot be greater than the Creator.
People who go to idols have brought themselves low because God has made us greater than those things. The Bible made us know that we are gods on Earth. It is people who are unaware of their true identity that go into worshiping of idols. God was very Direct about his zero tolerance for worshiping of idols.
Part of the commandments is that we should serve only God and have no other god except him. This is the commandment of God, going against it is going against His will.
The prayer of a dinner is an abomination unto God.