The value of any object is only as high as that with which we give it.
Precious metals are not solely for investment purposes, they have real world applications such as electronics, dental, chemical processes, hell even eating gold leaf.
I think a reason for the negative attitude is that gold is seen as an affluent material and one that can not be easily spent frivolously like cash. It is the ease of use of the "promissory notes" instead of carrying metals.