Personally I think the biggest concern is any situation where someone relies on AI output, but no one knows how the AI generated that particular output.

Imagine you want to travel to another country.  You get to the airport, everything is fine, you get your passport scanned and you are declined.  

A human officer would be able to explain that you're declined for a particular reason... but if the decline came from an AI system, then no one would know why you were declined.  Was the AI broken, was it a good reason?  Who knows?  But the impact is that you're now trapped (possibly even detained) without knowing why.

Similar situations might happen in the medical field where AI suggests a treatment that makes you worse. Did AI make a mistake or did something change?  

Of course, for lots of things, it doesn't really matter if we don't understand how it does something.  If AI generates a new story or a cool animation video, that's great... we don't need to know how it decided to create what it did.  For things that have a huge impact on humans (travel, medical, legal, etc) then we really need to know why AI is making specific suggestions.