The exchanges that holds your money holds your keys. If they go down you are subject to the systemic risk of the exchange.

There are some good youtube videos that go over Hot and Cold Storage for your keys.