Before answering this question, it is important to understated that files are stored in the hard disk of your computer, which is type of a magnetic data storage system.  

When you delete a file from your computer (even permanently), the file doesn't actually get s deleted. What this means is that the file is actually kept where it was, but now the computer just doesn't care anymore if it overwrites it or not. 

In fact, the modern firmware makes it so that the newly saved data is saved by overwriting the the oldest of the 'deleted' data. This is done so that the the recently 'deleted' data can be extracted easily and as cleanly as possible, if you need to recover it.

So essentially when you delete a file, you are just making your computer forget the address of the where it stored that file.