Don't believe pyrite has a taste, no, but given how visually distinct it is, shouldn't usually be a problem.